-- Logs begin at Mon 2022-06-20 05:14:40 UTC, end at Mon 2022-06-20 06:53:09 UTC. -- Jun 20 05:47:43.153009 dv-cinder-804757-29-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 20 05:47:43.162995 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jun 20 05:47:43 2022] *** Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: nodename: dv-cinder-804757-29-iscsi Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: machine: x86_64 Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: clock source: unix Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: pcre jit disabled Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: detected number of CPU cores: 4 Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: current working directory: / Jun 20 05:47:43.164527 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: detected binary path: /usr/bin/uwsgi-core Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: your processes number limit is 47680 Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: your memory page size is 4096 bytes Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: detected max file descriptor number: 2048 Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: lock engine: pthread robust mutexes Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: thunder lock: enabled Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 20 05:47:43.165142 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 20 05:47:43.209197 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: Python main interpreter initialized at 0x564b291c39c0 Jun 20 05:47:43.209197 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: python threads support enabled Jun 20 05:47:43.209197 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: your server socket listen backlog is limited to 100 connections Jun 20 05:47:43.209197 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: your mercy for graceful operations on workers is 90 seconds Jun 20 05:47:43.209583 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: mapped 403077 bytes (393 KB) for 2 cores Jun 20 05:47:43.209658 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: *** Operational MODE: preforking *** Jun 20 05:47:43.209897 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: *** uWSGI is running in multiple interpreter mode *** Jun 20 05:47:43.209897 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: spawned uWSGI master process (pid: 80157) Jun 20 05:47:43.210095 dv-cinder-804757-29-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 20 05:47:43.215277 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: spawned uWSGI worker 1 (pid: 80160, cores: 1) Jun 20 05:47:43.215277 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: spawned uWSGI worker 2 (pid: 80161, cores: 1) Jun 20 05:47:43.215277 dv-cinder-804757-29-iscsi devstack@keystone.service[80157]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 20 05:47:48.596114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 05:47:48.596114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Jun 20 05:47:48.596904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 05:47:48.596904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: from cryptography.utils import int_from_bytes Jun 20 05:47:48.597330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 05:47:48.597330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: from cryptography.utils import int_from_bytes Jun 20 05:47:48.597430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 20 05:47:48.597430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: from cryptography.utils import int_from_bytes Jun 20 05:47:48.645012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 20 05:47:48.645207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 20 05:47:48.645402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] command line args: [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 20 05:47:48.645493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 20 05:47:48.645674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 20 05:47:48.645754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 20 05:47:48.645854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ================================================================================ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 20 05:47:48.645941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] command line args: [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 20 05:47:48.646108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] admin_token = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.646214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 20 05:47:48.646354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_dir = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.646432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ================================================================================ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 20 05:47:48.646542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.646631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.646710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] config_source = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.646873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_dir = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.646957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] debug = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] config_source = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] debug = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.647957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_config_append = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.648980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_dir = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_config_append = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_dir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.649982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.650933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651350 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.651868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.652856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] publish_errors = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.45/identity {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] publish_errors = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.653965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] standard_threads = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] standard_threads = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.654965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_journal = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_json = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_stderr = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_journal = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] use_syslog = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_json = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.655965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.656045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_stderr = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.656045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] use_syslog = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.656180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.656280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 20 05:47:48.656365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.656533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.656620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.656772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.656857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.657030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.657108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.657385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.658873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.659843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.660959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.661921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.662976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.663919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.664858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.665956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.666945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.667904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.668976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.669933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.670896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.671875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.672978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.673956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.674894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.675932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.676871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.677992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.678989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.679942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.680894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.external = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.external = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.681988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.password = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] auth.token = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.password = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.682947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] auth.token = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.683996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.684923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.685881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.686857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.687898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.688903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.689859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.690995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.691963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.692983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.693928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.694847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.695854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696246 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.696946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.697954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.698918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.699874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.700957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.701976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.702922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.703881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.704026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.704583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.704782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.704861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.705953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.706955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.707954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.708891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.709976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.710999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.711944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.712883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.713999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.714943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.715985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.716973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.717925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.718970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.719056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.719214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.719454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.719537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.719692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.719799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.720496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.720597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.720750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.720830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.720988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.721950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.722986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.723963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724426 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.724887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.725838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.726955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727294 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.727993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.728920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.729971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.730903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.731886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732350 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.732968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.733037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.733136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80161) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 20 05:47:48.733218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.733425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 20 05:47:48.733595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80160) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 20 05:47:48.762951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.763094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.763177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.763311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.763424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.763542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.772395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.772573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.800225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.800339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.800451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.800536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.942965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.944146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.991650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.991879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.991971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:48.992210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.055990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.056074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.056208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.056304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.056422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.081912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.082050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.082134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.082269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.082351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.082486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.082569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.093357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.093612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.106657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.106888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.106978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.107885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.108016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80160) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.108098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.108270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80161) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 20 05:47:49.125611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.125821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 05:47:49.126179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.126262 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.126388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 05:47:49.126491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 05:47:49.126633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.126716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.126887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.127012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 20 05:47:49.127140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.127287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.127415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.127498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.127673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.127927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.138143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.138417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.138683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.138764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.138962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.139057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.139231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.139531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.147218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.147827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.150491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.150769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.151360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.151984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.152956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.153044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.153216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.153295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.153483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.153747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.180649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.181074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.182251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.182528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.182711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.182987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.187774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.188171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.188254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.188449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.189097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.189380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.196240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.196483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.196566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.196833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.200952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.201241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.201324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.201525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.201606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.201806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.201890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.202965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.203152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.222071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.222209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.222348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.222730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.227227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.227371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.230845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.230929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.231243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.231340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.236271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.236360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.239904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.239989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.243553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.243648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.243851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.243934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.244959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.245049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.264317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.264410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.264595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.264677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.264857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.264952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.265127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.265211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.273516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.273611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.275210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.275311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.275507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.275601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.278689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.278781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.278968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.279957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.293039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.293124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.293316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.293399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.293606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.293687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.293884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.293966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.295403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.295486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.295693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.295774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.295978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.296060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.314602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.314735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.317875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.318026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.321155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.321330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.322872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.323063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.324529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.324721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.324803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.324997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.325081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.325273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.332356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.332571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.333948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.334170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.337158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.337398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.337483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.337688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.337820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.338079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.338170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.338361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.347080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.347392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.347724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.347923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.348019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.348212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.355728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80160) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.356004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.356087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80161) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 20 05:47:49.356279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.356362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.356545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.356628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.356809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.356892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.357984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358075 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.358872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.359051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.359133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.359346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80160) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.359431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.359626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80161) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 20 05:47:49.397631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564b291c39c0 pid: 80160 (default app) Jun 20 05:47:49.398562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x564b291c39c0 pid: 80161 (default app) Jun 20 05:47:49.400985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4cecd-3948-4fdc-b037-012382d8ad30 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:49.401279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4cecd-3948-4fdc-b037-012382d8ad30 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:49.401558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4cecd-3948-4fdc-b037-012382d8ad30 None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:49.405705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1/1] 10.222.0.45 () {50 vars in 791 bytes} [Mon Jun 20 05:47:49 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:47:57.024563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:57.025125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:57.025125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:57.029421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1/2] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:47:57 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:47:57.035465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:57.035852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:57.036034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:57.104428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: INFO dbcounter [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] Registered counter for database keystone Jun 20 05:47:57.124972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG oslo_db.sqlalchemy.engines [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80160) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 20 05:47:57.131042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writer thread running {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 20 05:47:57.305024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80160) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 20 05:47:57.305438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG passlib.handlers.bcrypt [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80160) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 20 05:47:57.359562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:47:57.361115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:47:57.478917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 2/3] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:47:57 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 20 05:47:57.596858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:47:57.666043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: INFO dbcounter [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] Registered counter for database keystone Jun 20 05:47:57.685105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG oslo_db.sqlalchemy.engines [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80161) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 20 05:47:57.691857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writer thread running {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 20 05:47:57.809232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010820e-3afe-4c8d-9dda-e7d01a9203b8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:47:57.813954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:47:57.814606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:57.815363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:57.815643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:57.824062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:47:57.875032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 20 05:47:57.875428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80da584-ea23-42ea-8a69-3aec6c48bc54 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:47:57.885484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 2/4] 10.222.0.45 () {58 vars in 1324 bytes} [Mon Jun 20 05:47:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 05:47:57.891380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:47:57.905457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b790b58c-bdfa-4786-9d73-3d2941feecae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:47:57.911713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:47:57.912864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:57.913153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:57.913431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] PATH_INFO: `/v3/projects/b72f99dc88224a27868e009fcc68d646` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:57.917554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] RBAC: Authorizing `identity:get_project(project_id=b72f99dc88224a27868e009fcc68d646)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:47:57.973067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING keystone.common.rbac_enforcer.enforcer [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 20 05:47:57.973419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54471c13-05ae-4492-9ab7-2c79fa761628 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:47:57.980567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 3/5] 10.222.0.45 () {56 vars in 1207 bytes} [Mon Jun 20 05:47:57 2022] GET /identity/v3/projects/b72f99dc88224a27868e009fcc68d646 => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:47:59.949005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2cd2-8453-4c17-a4f6-ebb8b445ef72 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:59.949506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2cd2-8453-4c17-a4f6-ebb8b445ef72 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:59.949598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f2cd2-8453-4c17-a4f6-ebb8b445ef72 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:59.954241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 3/6] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:47:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:47:59.959512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:47:59.959926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:47:59.960085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:47:59.979604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:47:59.981127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:47:59.988892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 4/7] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:47:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 20 05:48:00.010943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f2cd2-8453-4c17-a4f6-ebb8b445ef72 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:00.023427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f2cd2-8453-4c17-a4f6-ebb8b445ef72 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:00.027648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caced1e2-6e60-4c1f-aba5-607dd4898253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:00.028281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced1e2-6e60-4c1f-aba5-607dd4898253 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:00.028569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced1e2-6e60-4c1f-aba5-607dd4898253 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:00.028849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caced1e2-6e60-4c1f-aba5-607dd4898253 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:00.037136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caced1e2-6e60-4c1f-aba5-607dd4898253 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:00.038893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caced1e2-6e60-4c1f-aba5-607dd4898253 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:00.047558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 4/8] 10.222.0.45 () {58 vars in 1324 bytes} [Mon Jun 20 05:48:00 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 05:48:00.052677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:00.061499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92174ab3-f8c9-4f04-ae7a-dffae0095a0c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:00.065847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d84d5-ee98-4c79-9ddb-579404e39dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:00.066703 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d84d5-ee98-4c79-9ddb-579404e39dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:00.066989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d84d5-ee98-4c79-9ddb-579404e39dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:00.067291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d84d5-ee98-4c79-9ddb-579404e39dd4 admin admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:00.068696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d84d5-ee98-4c79-9ddb-579404e39dd4 admin admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:00.070509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d84d5-ee98-4c79-9ddb-579404e39dd4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:00.072142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 5/9] 10.222.0.45 () {56 vars in 1198 bytes} [Mon Jun 20 05:48:00 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:02.135290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a0f7b-665b-4906-9e94-7106fc341bda None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.135817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a0f7b-665b-4906-9e94-7106fc341bda None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.135817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a0f7b-665b-4906-9e94-7106fc341bda None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.137172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 5/10] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:02.142774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.143198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.143380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.160064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:02.161234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:02.176375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1c48-9d15-4e3a-a429-b0b5c3b569fe None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.176677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1c48-9d15-4e3a-a429-b0b5c3b569fe None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.176961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf1c48-9d15-4e3a-a429-b0b5c3b569fe None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.178318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 6/11] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:02.183311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.183610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.183897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.223160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG passlib.handlers.bcrypt [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80161) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 20 05:48:02.223592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG passlib.handlers.bcrypt [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80161) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 20 05:48:02.242214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 6/12] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:02.265777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.269814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:02.271432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:02.287877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.365726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34d5e9-0f95-4860-84e1-22270c91cde0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.368380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 7/13] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:02.391155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.398518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:02.399347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.399638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.399919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.411611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:02.422767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:02.423772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:02.424586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 7/14] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:02.429765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.489707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9079139-7b6d-4c41-a2f0-cf61c2ff8bfc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.491991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.509945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.527035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:02.527956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.528239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.528369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc892e-c058-4ed5-b21e-89045eda7980 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.528513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.562245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:02.563463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.563756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.564041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.564793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:02.565107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:02.566534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:02.578847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 8/15] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:02.584336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.594537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:02.596104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:02.681266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826c535-f9c0-4cef-a758-1e2c35533fe4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.691832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 8/16] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 05:48:02.697540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.700152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.713525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:02.714397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.714681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.714961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.716185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:02.717626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:02.719800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 9/17] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:02.725210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.739584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.793495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5d89f-0cdb-46fd-bf36-19fce16944ac None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.825701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a0e2df-abcf-41b6-abe2-ecd14e0e03c2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.826632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:02.827473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.827763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.828049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.846950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:02.848716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:02.860968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:02.861754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:02.862040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:02.862317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:02.863544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:02.864982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:02.866566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 10/18] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:48:02.871491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:02.872365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:02.873162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:48:02.873162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:48:02.873162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:48:02.873162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:02.873361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.873425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 9/19] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:02.878061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:02.933857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.952163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:02.970980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012373a-4d85-48fc-8017-7c8d811acc50 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:02.977413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd08d683-53c3-4e67-af92-c5d40785541c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:03.004412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:03.005224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:03.005799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:03.006102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:03.006938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.domain.id=default, user.id=ee104dedf13a4c7aa48858230c010f93) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:03.007263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:03.008723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9777cf6-98ab-4492-b344-988b8472edb1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:03.010004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:03.010655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:03.010941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:03.011219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:03.011969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:03.012278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:03.013676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235d848c-1544-4121-bbe0-4280a522aa29 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:03.015390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 11/20] 10.222.0.45 () {56 vars in 1322 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/role_assignments?user.id=ee104dedf13a4c7aa48858230c010f93&scope.domain.id=default&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:03.033205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 10/21] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:05.368889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf64b-c610-4ff9-a9d6-1c2b888a7380 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.369396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf64b-c610-4ff9-a9d6-1c2b888a7380 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.369467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbf64b-c610-4ff9-a9d6-1c2b888a7380 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.370815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 12/22] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:05.376691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.377087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.377252 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.382766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0a9a2-0761-47b5-8ee8-7e265f1c1b53 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.383347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0a9a2-0761-47b5-8ee8-7e265f1c1b53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.383852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0a9a2-0761-47b5-8ee8-7e265f1c1b53 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.385399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 13/23] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:05.390578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.391110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.391641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.404272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:05.405497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:05.410781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:05.412235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:05.489966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 11/24] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:05.517605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 14/25] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:05.522647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:05.564672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:05.592949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:05.695437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:05.714099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c447bbd2-055d-4a21-b4ac-86af12f86d31 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:05.725794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f256f-9cdc-4e09-8e5f-131cfb443c2b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:05.772542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9e38e-101b-4dee-ab93-11a4ea749601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:05.774886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e38e-101b-4dee-ab93-11a4ea749601 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.775409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e38e-101b-4dee-ab93-11a4ea749601 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.776723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9e38e-101b-4dee-ab93-11a4ea749601 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.792283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:05.795227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.795747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.796240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.798188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9e38e-101b-4dee-ab93-11a4ea749601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e3f9e38e-101b-4dee-ab93-11a4ea749601 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:05.806104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:05.806931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:05.807706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:05.825177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 15/26] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:05.826919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06d7b1-cd7d-444a-8a78-9e8860f64d99 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.828018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:05.829726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:05.830215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06d7b1-cd7d-444a-8a78-9e8860f64d99 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.830712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06d7b1-cd7d-444a-8a78-9e8860f64d99 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.837070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 16/27] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:05.843814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:05.844107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:05.844392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:48:05.874760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:05.876240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:05.877039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:48:05.877039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:48:05.877039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:48:05.877039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:05.877250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:05.877773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:05.880953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 12/28] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:05.881664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:05.998397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 17/29] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:05.998987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.043889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:06.049155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1db8a4-2def-4b02-8dc2-442ca9ae5060 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.090376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:06.092374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.093054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.093416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.093771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.094636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:06.094950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.097219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b0e156-5ec4-45da-9ebd-5a5ae660019b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.118161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 13/30] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:06.118161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00eef78-9d41-4664-b4ca-6e19d54c7903 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.118161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00eef78-9d41-4664-b4ca-6e19d54c7903 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.118161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00eef78-9d41-4664-b4ca-6e19d54c7903 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.118161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 14/31] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:06.122625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00eef78-9d41-4664-b4ca-6e19d54c7903 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.142948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de6142-1312-495d-baca-12e8a2ebff05 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.183718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.184850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.185449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.186029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.187013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:06.188067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.189969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d553afd-d9b1-4778-9c99-6e4a58b35052 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.209411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 18/32] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:06.215405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.215960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.216466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.235608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:06.237185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:06.246300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00eef78-9d41-4664-b4ca-6e19d54c7903 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.290374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.290970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.291296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.291564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.292589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.294226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.317224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 19/33] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:06.318557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.393199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:06.443886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca600a-5024-4bc0-b866-b2338dbd717a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.471462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 15/34] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Jun 20 05:48:06 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:06.479730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.498465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.499407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.499695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.500011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.502285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.503831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.505568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 20/35] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:06 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:06.506346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.519103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:06.617144 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e19c2ce-beb2-4a1b-8355-b78efa234e5c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.662803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.663902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.664400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.664897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.666119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.668148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.704586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 16/36] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:48:06 2022] POST /identity/v3/projects => generated 301 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:06.711541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83b5c1-7800-43c5-ad27-b25eddfa44ef None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.712960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.739902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:06.755277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760912d-cb20-4fcb-ab23-c681af752103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.755277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760912d-cb20-4fcb-ab23-c681af752103 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.755277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760912d-cb20-4fcb-ab23-c681af752103 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.755645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760912d-cb20-4fcb-ab23-c681af752103 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.756466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760912d-cb20-4fcb-ab23-c681af752103 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.758141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8760912d-cb20-4fcb-ab23-c681af752103 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.860064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:06.918215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518eb545-27ae-4019-9b72-238ede75b74d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:06.961467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80f203-7b83-48c9-906d-ef14b2b4f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:06.962543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80f203-7b83-48c9-906d-ef14b2b4f7ef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.963066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80f203-7b83-48c9-906d-ef14b2b4f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.963536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80f203-7b83-48c9-906d-ef14b2b4f7ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.964988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80f203-7b83-48c9-906d-ef14b2b4f7ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:06.966547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 21/37] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Jun 20 05:48:06 2022] POST /identity/v3/roles => generated 225 bytes in 460 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:06.966769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80f203-7b83-48c9-906d-ef14b2b4f7ef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:06.968285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bbc9-ed20-49b7-a6a7-2ce211ba80ea None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.969125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bbc9-ed20-49b7-a6a7-2ce211ba80ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.969415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bbc9-ed20-49b7-a6a7-2ce211ba80ea None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:06.970179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 17/38] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:06.971082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 22/39] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:06.976478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96bbc9-ed20-49b7-a6a7-2ce211ba80ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:06.977541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdace4-8f21-4777-ad98-66d76f3af809 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:06.977982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdace4-8f21-4777-ad98-66d76f3af809 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:06.978415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afdace4-8f21-4777-ad98-66d76f3af809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.008070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2afdace4-8f21-4777-ad98-66d76f3af809 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:07.010025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2afdace4-8f21-4777-ad98-66d76f3af809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:07.105463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96bbc9-ed20-49b7-a6a7-2ce211ba80ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:07.115275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 18/40] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:07.115275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648d13-10a0-42bd-a887-0405982ab634 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:07.115275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648d13-10a0-42bd-a887-0405982ab634 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:07.115275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2648d13-10a0-42bd-a887-0405982ab634 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.115275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 19/41] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:07.115751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:07.115827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:07.116237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.137846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:07.139071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:07.159480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:07.160372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:07.160646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:07.160927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] PATH_INFO: `/v3/domains/default/users/ee104dedf13a4c7aa48858230c010f93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.163093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ee104dedf13a4c7aa48858230c010f93, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:07.164706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:07.183861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 23/42] 10.222.0.45 () {58 vars in 1361 bytes} [Mon Jun 20 05:48:06 2022] PUT /identity/v3/domains/default/users/ee104dedf13a4c7aa48858230c010f93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:07.187784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:07.221565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:07.228632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 20/43] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:07.244541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:07.254393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:07.377464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b904b60e-fa8a-48d7-9dcc-cc1c0d3f242d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:07.416387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:07.417190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:07.417478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:07.417759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.418947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:07.420459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:07.422127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 21/44] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:07.423482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49b4de9-099c-445e-9015-a2c22f890ee3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:07.427553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:07.461153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b32c3b-f812-4456-bca9-2518ce3e430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:07.461744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b32c3b-f812-4456-bca9-2518ce3e430a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:07.462041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b32c3b-f812-4456-bca9-2518ce3e430a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:07.462325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b32c3b-f812-4456-bca9-2518ce3e430a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.463077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b32c3b-f812-4456-bca9-2518ce3e430a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:07.464687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b32c3b-f812-4456-bca9-2518ce3e430a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:07.481242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 24/45] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Jun 20 05:48:07 2022] POST /identity/v3/roles => generated 223 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:07.500764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:07.539984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321d2b50-fd28-4492-846a-04df9094fca0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:07.578772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6dcab-8400-4c8b-a618-b5cbae017f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:07.579486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6dcab-8400-4c8b-a618-b5cbae017f8d None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:07.579777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6dcab-8400-4c8b-a618-b5cbae017f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:07.580060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6dcab-8400-4c8b-a618-b5cbae017f8d None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:07.580790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6dcab-8400-4c8b-a618-b5cbae017f8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:07.582262 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6dcab-8400-4c8b-a618-b5cbae017f8d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:07.611511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 22/46] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:48:07 2022] POST /identity/v3/projects => generated 312 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:48:09.206329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f7940-e030-4706-af2a-3c47b8d83be9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.206869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f7940-e030-4706-af2a-3c47b8d83be9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.206869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428f7940-e030-4706-af2a-3c47b8d83be9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.208291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 25/47] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:09.213659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.214075 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.214241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.231431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:09.232630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:09.297401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 23/48] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:09.320644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f7940-e030-4706-af2a-3c47b8d83be9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:09.419745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428f7940-e030-4706-af2a-3c47b8d83be9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:09.421259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:09.455825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:09.456675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.456917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.457197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.467578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:09.475554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:09.476372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:09.477374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 26/49] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:09.482467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:09.530811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:09.581325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c184a7-dd9c-40a8-8dc0-c48c6f7acb8d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:09.615153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:09.615858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.616152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.616437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.617194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:09.617508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:09.618895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e9c446-b8d9-4ea1-9e62-dfda64408ebf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:09.627490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 24/50] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:09.633039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:09.635551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c32d-04cb-470d-923a-b89ead19ee10 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.635891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c32d-04cb-470d-923a-b89ead19ee10 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.636181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072c32d-04cb-470d-923a-b89ead19ee10 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.637479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 25/51] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:09.642282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.642618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.642911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.740537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ac8440-4089-4bb9-ada5-1b4f4d900088 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:09.763437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:09.764664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:09.774787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:09.775712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.775996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.776277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.777487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:09.778948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:09.780609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 27/52] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:09 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:09.785930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:09.816552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:09.832868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 26/53] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:09.856536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:09.886847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257558c9-22cb-46d8-8d2a-2e56bcd02873 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:09.899237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:09.920340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:09.921140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.921463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.921770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.922957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:09.924471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:09.926096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 28/54] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:09.931743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:09.956450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7e680-ef55-4009-9d25-fba4b0277005 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:09.989777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:09.990626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:09.990871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:09.991158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:09.992379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:09.993840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:09.995502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 27/55] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:10.001570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:10.027615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:10.031512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97612978-b25e-4ae5-a6dd-c0a0fcd5029f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:10.064262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:10.064915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:10.065205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:10.065489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:10.066325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.domain.id=default, user.id=ee104dedf13a4c7aa48858230c010f93) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:10.066643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:10.068150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932ea89-317f-4a88-bbc4-c69c5dd6b1a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:10.074183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 29/56] 10.222.0.45 () {56 vars in 1322 bytes} [Mon Jun 20 05:48:09 2022] GET /identity/v3/role_assignments?user.id=ee104dedf13a4c7aa48858230c010f93&scope.domain.id=default&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:10.114781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e1173-c5b9-4c2d-b82c-94a4a010b3d9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:10.127085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:10.151795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda4061-dbeb-46e1-a646-582900b31952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:10.152480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4061-dbeb-46e1-a646-582900b31952 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:10.152780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4061-dbeb-46e1-a646-582900b31952 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:10.153066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda4061-dbeb-46e1-a646-582900b31952 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:10.153801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda4061-dbeb-46e1-a646-582900b31952 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:10.155315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda4061-dbeb-46e1-a646-582900b31952 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:10.181412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 28/57] 10.222.0.45 () {60 vars in 1140 bytes} [Mon Jun 20 05:48:10 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:48:12.196465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7897f-88c1-402a-b662-7b51ba3e125c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:12.196951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7897f-88c1-402a-b662-7b51ba3e125c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:12.197031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7897f-88c1-402a-b662-7b51ba3e125c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:12.198414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 30/58] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:12.203585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:12.203993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:12.204152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:12.220761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:12.222195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:12.285126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 29/59] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:12.309680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7897f-88c1-402a-b662-7b51ba3e125c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:12.408658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7897f-88c1-402a-b662-7b51ba3e125c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:12.440380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac1e34-b8f8-4570-8cdc-cd2fa7731baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:12.441155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1e34-b8f8-4570-8cdc-cd2fa7731baf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:12.441440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1e34-b8f8-4570-8cdc-cd2fa7731baf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:12.441716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac1e34-b8f8-4570-8cdc-cd2fa7731baf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:12.442883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac1e34-b8f8-4570-8cdc-cd2fa7731baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:12.444360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac1e34-b8f8-4570-8cdc-cd2fa7731baf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:12.445943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 31/60] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:12.451280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:12.537299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:12.546888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be490ac2-057d-41d7-af96-8912d8c1a587 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:12.579988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37286597-6f06-47f2-ab8d-224b9942c234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:12.580657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37286597-6f06-47f2-ab8d-224b9942c234 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:12.580945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37286597-6f06-47f2-ab8d-224b9942c234 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:12.581226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37286597-6f06-47f2-ab8d-224b9942c234 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:12.581932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37286597-6f06-47f2-ab8d-224b9942c234 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:12.583393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37286597-6f06-47f2-ab8d-224b9942c234 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:12.608011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 30/61] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:12 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:15.759601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d745f5-14e0-494b-9cbd-300a3027f7d9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:15.759601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d745f5-14e0-494b-9cbd-300a3027f7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:15.759601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d745f5-14e0-494b-9cbd-300a3027f7d9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:15.759601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 32/62] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:15.772039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:15.772467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:15.773924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:15.800295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:15.801751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:15.911728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241751c-ee04-4425-b491-a684833cd85f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:15.912360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241751c-ee04-4425-b491-a684833cd85f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:15.912890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6241751c-ee04-4425-b491-a684833cd85f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:15.914487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 33/63] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:15.921093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759ff0-c759-415f-b7fa-17d8cd9dabac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:15.922100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759ff0-c759-415f-b7fa-17d8cd9dabac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:15.922578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759ff0-c759-415f-b7fa-17d8cd9dabac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:15.930959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 31/64] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:15.946428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f7759ff0-c759-415f-b7fa-17d8cd9dabac None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:15.949051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f7759ff0-c759-415f-b7fa-17d8cd9dabac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:15.961354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.027723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.123712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 34/65] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:16.125580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81f28-951a-4f9d-bc9b-d9e3141a2ac3 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.125885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81f28-951a-4f9d-bc9b-d9e3141a2ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.126176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c81f28-951a-4f9d-bc9b-d9e3141a2ac3 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.127629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 35/66] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:16.133661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.133957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.134245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.161838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.164713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:16.165927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:16.173612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0fd20-d35f-4f98-a063-bf2c56ad666e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.212786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fea5af-b0b7-4bb1-a3c4-7db7c01d9f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.213610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea5af-b0b7-4bb1-a3c4-7db7c01d9f97 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.213903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea5af-b0b7-4bb1-a3c4-7db7c01d9f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.214189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fea5af-b0b7-4bb1-a3c4-7db7c01d9f97 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.227573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fea5af-b0b7-4bb1-a3c4-7db7c01d9f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:16.236320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 36/67] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-19fea5af-b0b7-4bb1-a3c4-7db7c01d9f97 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:16.236633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:16.237710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:16.238635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:16.239372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.239448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 32/68] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:16.239448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3799b-7409-4ef9-86f8-11c1f876a954 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.239448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3799b-7409-4ef9-86f8-11c1f876a954 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.239654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d3799b-7409-4ef9-86f8-11c1f876a954 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.247387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 33/69] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:16.247387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.247387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.247387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.266080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:16.267926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:16.331102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.341461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e812294-e2d8-435c-9ae1-a04d2d57b997 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.344454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 34/70] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:16.347440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.384108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.384964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.385256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.385589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.397349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:16.406154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:16.406999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:16.407808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 37/71] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:16.408432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.420413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.454420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e553418-1a2d-40af-b19f-e190f8112506 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.491119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.491832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.492124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.492405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.493515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:16.493842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:16.495586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:16.504818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 35/72] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:16.504818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.522518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f41c0e-0eb0-46e6-a5b8-877c01becdba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.557492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.558306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.558592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.558870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.570353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:16.578877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:16.579736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:16.580483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 38/73] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:16.581100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.593125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.612313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b13533-b8b9-4f95-876e-d6c7a7a57752 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.640077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.646330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.647129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.647450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.647731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.658373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:16.666087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:16.666929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:16.667710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 36/74] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:16.668320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.685000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8495e2-1ec0-4bb2-afbb-d4434dca9fd4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.717351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.718547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.718839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.719121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.719899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:16.720213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:16.721861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5646c-43a6-4d52-b318-49b27cf59280 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:16.730249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 39/75] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:16.731600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1eaf5-b0f1-43a7-9d08-c0bb7a9a1c0d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.731901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1eaf5-b0f1-43a7-9d08-c0bb7a9a1c0d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.732178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1eaf5-b0f1-43a7-9d08-c0bb7a9a1c0d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.733436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 40/76] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:16.734802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1eaf5-b0f1-43a7-9d08-c0bb7a9a1c0d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.768189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc4065b-86a9-4884-b9b5-22ae81bbe6e8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.800005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.800885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.801172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.801450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.802928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.820084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:16.820084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:16.821284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 37/77] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:16.821940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.842542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1eaf5-b0f1-43a7-9d08-c0bb7a9a1c0d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.870731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:16.877831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.878503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.878796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.879078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.879872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:16.880187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:16.881578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:16.889470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 41/78] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:16.890803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:16.925303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d575662f-1896-4998-9f9a-fd459f35a574 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.958298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:16.958977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.959290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.959587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.960341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:16.960658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:16.962033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89a5078-722c-43bf-93d6-77d51ca40508 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:16.970116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 38/79] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:16.971478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:16.972163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:16.972575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:16.987075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ea0c1e-b60c-4edd-bc77-9147c872df95 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:16.988463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:16.989665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:17.019886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.020756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.021045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.021325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.022502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.023981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.025596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 42/80] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:16 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:17.031277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.033778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.054932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 39/81] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:17.063411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.096872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.141259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962a8541-5f4b-4405-964e-e97e008ed0b9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.155086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3f0af-c2f8-44c0-8e02-0938a45e011e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.173824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.174479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.174773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.175060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.189577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.190456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.190747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.191039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.192268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.193730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.195388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 40/82] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:17.196231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.244920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.246379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.248248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.291507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d109c481-8d05-4a21-9076-f44c0dd405d8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.301126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.325900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.326769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.327061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.327896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.329086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.330523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.332211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 41/83] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:17.332980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.345545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 43/84] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:17.346435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.433168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186b51d-4bdd-49aa-93cb-91d879d6d79c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.441450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f295e-10b6-4e7f-95b9-e2497dac4d68 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.442940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.466480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.467131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.467458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.467763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.477132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.477426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.477713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.478890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.480400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.482184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 44/85] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:17.482870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.516020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.537973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.539475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.579965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5c3af-27d6-4593-9931-99dcf378e0c8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.617193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.617849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.618137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.618424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.635951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 42/86] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:17.636819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.657837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.685991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.687999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.710362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.733989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47a038b-32d5-40f5-83be-d99531b510b4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.766899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.767568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.767854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.768367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.787866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 45/87] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:17.787866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.838393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.839918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.853384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.883821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef99df7-a72d-4448-a0bb-acdfcafaa4bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:17.913951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:17.918246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:17.919106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:17.919416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:17.919697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:17.920900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:17.922366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:17.924084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 46/88] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:17.924820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:17.938469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 43/89] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:17.939206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.023271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b95d77-9c4c-40fe-93cf-ab2b1573e7a7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.036414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067e06f5-6e2b-4485-ae22-6e6c5ea39fe8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.056421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.057092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.057386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.057667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.058387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.059892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.070998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.071897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.072190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.072474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.072832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:18.073709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.075408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.078219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 44/90] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:17 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:18.079069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.091325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 47/91] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:17 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:18.092436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.127528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:18.198830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e6f3e-f7af-4485-8107-d9143f484707 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.212985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917049f4-0b64-4e14-b214-eda391642006 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.238651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.239575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.239868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.240156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.241431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.242889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.244690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 45/92] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:18 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:18.249504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.257506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.258173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.258461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.258746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.259747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=31594a42591343ab884a5efd018433c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:18.260066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.261551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.269333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 48/93] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:18 2022] GET /identity/v3/role_assignments?user.id=31594a42591343ab884a5efd018433c4&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:18.270149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.358671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:18.388526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0543e0a7-e6ef-44d9-8e5e-da76ca95fca8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.409364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:18.439745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.440907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.441357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.441644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] PATH_INFO: `/v3/projects/f96f145b0e1b4199b98766672d83163d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.443520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] RBAC: Authorizing `identity:get_project(project_id=f96f145b0e1b4199b98766672d83163d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.445123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.447105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 46/94] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:18 2022] GET /identity/v3/projects/f96f145b0e1b4199b98766672d83163d => generated 312 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:18.448103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.453648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3511038-62a0-4bda-b412-b75bd747d4b2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.500644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.501316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.501609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.501894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.502799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=ee104dedf13a4c7aa48858230c010f93) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:18.503060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.504577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.511153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 49/95] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:18 2022] GET /identity/v3/role_assignments?user.id=ee104dedf13a4c7aa48858230c010f93&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:18.512119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:18.604500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430c11d-f6bc-4440-a094-364e733f6b0c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.642851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.643629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.643927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.644348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.645213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] RBAC: Adding query filter params (role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=31594a42591343ab884a5efd018433c4) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:18.645535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.646993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f2a8d-f44d-4754-87f4-918e2633a523 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.656973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 47/96] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:18 2022] GET /identity/v3/role_assignments?user.id=31594a42591343ab884a5efd018433c4&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=df14633e5004446c8c52b442c7d11a91 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:18.720531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960de0c0-d182-4baa-bc2c-2e164226e5aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:18.761576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:18.775903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:18.776574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:18.776867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:18.777155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:18.778006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=f96f145b0e1b4199b98766672d83163d, user.id=31594a42591343ab884a5efd018433c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:18.778330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:18.779842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd53d83-0006-46ba-b3d4-c61190144bd5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:18.786933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 50/97] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:18 2022] GET /identity/v3/role_assignments?user.id=31594a42591343ab884a5efd018433c4&scope.project.id=f96f145b0e1b4199b98766672d83163d&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:20.737429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fc64c-c63a-48dd-8d1c-2ab33b0db33f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:20.738554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fc64c-c63a-48dd-8d1c-2ab33b0db33f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:20.739194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394fc64c-c63a-48dd-8d1c-2ab33b0db33f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:20.740895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 48/98] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:20.746370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:20.747263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:20.747770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:20.772214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:20.773724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:20.869941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 51/99] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:20.908259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fc64c-c63a-48dd-8d1c-2ab33b0db33f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:20.967228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:21.037020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394fc64c-c63a-48dd-8d1c-2ab33b0db33f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:21.074837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360470af-e3ff-4eca-a314-7932722ec57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:21.075734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360470af-e3ff-4eca-a314-7932722ec57b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.076022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360470af-e3ff-4eca-a314-7932722ec57b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.076300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360470af-e3ff-4eca-a314-7932722ec57b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.096150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360470af-e3ff-4eca-a314-7932722ec57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-360470af-e3ff-4eca-a314-7932722ec57b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:21.104231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:21.105064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:21.105885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 49/100] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:21.112165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:21.148212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0bf97-1ffe-4527-88b8-f42ab855fd82 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.148663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0bf97-1ffe-4527-88b8-f42ab855fd82 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.149042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0bf97-1ffe-4527-88b8-f42ab855fd82 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.150647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 50/101] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:21.156318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.156715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.157097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.185439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:21.186846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:21.303113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74b9df-4695-4712-a5ca-b84002bf3390 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:21.318493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 51/102] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:21.343612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:21.352547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:21.379973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:21.380946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.381241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.381786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.382950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:21.383349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:21.384767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:21.401606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 52/103] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:21.408311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:21.416819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:21.542237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1115b7e-26e0-4cd0-ba3c-6b388cd84237 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:21.598768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6313c1ff-671e-49db-b739-6a963e2f6f7c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:21.603969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684423be-e4e6-4e31-857b-01f66f386fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:21.604829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684423be-e4e6-4e31-857b-01f66f386fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.605123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684423be-e4e6-4e31-857b-01f66f386fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.605401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684423be-e4e6-4e31-857b-01f66f386fd2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.616731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684423be-e4e6-4e31-857b-01f66f386fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-684423be-e4e6-4e31-857b-01f66f386fd2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:21.624731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:21.625610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:21.626805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 52/104] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:21.627677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da38c-5669-4ef3-8b6c-d66f0c141c1a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.627677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da38c-5669-4ef3-8b6c-d66f0c141c1a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.627801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470da38c-5669-4ef3-8b6c-d66f0c141c1a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.633227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 53/105] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:21.633227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d6f35-02ab-4859-bb4f-accf9b81c037 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.633227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d6f35-02ab-4859-bb4f-accf9b81c037 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.633227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d6f35-02ab-4859-bb4f-accf9b81c037 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.633227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 54/106] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:21.633227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d6f35-02ab-4859-bb4f-accf9b81c037 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:21.656238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e5c94-4687-4ea6-8551-0ff2b0ac57d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:21.657125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e5c94-4687-4ea6-8551-0ff2b0ac57d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.657410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e5c94-4687-4ea6-8551-0ff2b0ac57d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.657688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e5c94-4687-4ea6-8551-0ff2b0ac57d6 None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.660047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e5c94-4687-4ea6-8551-0ff2b0ac57d6 None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:21.662665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e5c94-4687-4ea6-8551-0ff2b0ac57d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:21.665510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 53/107] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:21.667022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87c07a-44cd-4cc4-b9a3-2ac8a5be8d5b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.667562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87c07a-44cd-4cc4-b9a3-2ac8a5be8d5b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.668052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f87c07a-44cd-4cc4-b9a3-2ac8a5be8d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.689985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9f87c07a-44cd-4cc4-b9a3-2ac8a5be8d5b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:21.691805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f87c07a-44cd-4cc4-b9a3-2ac8a5be8d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:21.745594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:21.752997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:21.757018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d6f35-02ab-4859-bb4f-accf9b81c037 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:21.784417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 54/108] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:21.785705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693cee-c6cf-44b0-b2d5-739477ea6d2d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.786005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693cee-c6cf-44b0-b2d5-739477ea6d2d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.786290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c693cee-c6cf-44b0-b2d5-739477ea6d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.803173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:21.804340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.804567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9c693cee-c6cf-44b0-b2d5-739477ea6d2d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:21.805049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.805542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.805776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c693cee-c6cf-44b0-b2d5-739477ea6d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:21.806777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:21.807334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:21.809469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:21.820171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 55/109] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:21.821348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:21.890632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 55/110] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:21.890632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5b017-50ba-49b2-b223-679813105c6a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.890632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5b017-50ba-49b2-b223-679813105c6a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.890632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5b017-50ba-49b2-b223-679813105c6a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.890632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 56/111] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:21.891290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5b017-50ba-49b2-b223-679813105c6a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:21.933983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f63c045-3f2e-48c6-bc42-7e7dbb2e9380 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:21.969087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:21.969971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:21.970264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:21.970549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:21.972103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:21.973691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:21.975610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 56/112] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:21.976692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.003686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.008638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5b017-50ba-49b2-b223-679813105c6a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.017208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.046240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee67a2e-25bf-452f-a221-f630cad6e025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.047041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee67a2e-25bf-452f-a221-f630cad6e025 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.047360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee67a2e-25bf-452f-a221-f630cad6e025 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.047637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee67a2e-25bf-452f-a221-f630cad6e025 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.058366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee67a2e-25bf-452f-a221-f630cad6e025 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5ee67a2e-25bf-452f-a221-f630cad6e025 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:22.067053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:22.068267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 57/113] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:22.074157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.074848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.074848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.087845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:22.088098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0a677-a463-431a-a6f3-3541686e7c7a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.089219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:22.126792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.127711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.128004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.128286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] PATH_INFO: `/v3/users/7e17af398e2241a6b3774f42beb395cf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.141455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=7e17af398e2241a6b3774f42beb395cf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.142847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.144578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 57/114] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:21 2022] GET /identity/v3/users/7e17af398e2241a6b3774f42beb395cf => generated 289 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:22.145339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.161099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 58/115] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:22.161983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.226598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.246516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538eac88-65a3-4e81-9450-28aba14b7ce3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.257208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.264371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68b701-b61f-46ed-8260-0a6afc7f80a9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.280801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.281600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.281887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.282163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.292259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:22.298067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf09669-2118-4c20-8291-c0c9feb72c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.298960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf09669-2118-4c20-8291-c0c9feb72c57 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.299274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf09669-2118-4c20-8291-c0c9feb72c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.299575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf09669-2118-4c20-8291-c0c9feb72c57 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/31594a42591343ab884a5efd018433c4/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:22.300427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:22.301395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 58/116] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:22.302363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.304352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf09669-2118-4c20-8291-c0c9feb72c57 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c253b10dbca4a15b11c1d888597ca3b, user_id=31594a42591343ab884a5efd018433c4, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.305811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf09669-2118-4c20-8291-c0c9feb72c57 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.319035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 59/117] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:22 2022] PUT /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/31594a42591343ab884a5efd018433c4/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:22.320338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4c16-c172-4e08-871e-574e92fbc352 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.320632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4c16-c172-4e08-871e-574e92fbc352 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.320909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124a4c16-c172-4e08-871e-574e92fbc352 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.322471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 60/118] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:22.323616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a4c16-c172-4e08-871e-574e92fbc352 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.427848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111c601-ce98-4772-b5e1-9537eec00030 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.442289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124a4c16-c172-4e08-871e-574e92fbc352 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.467517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.467517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.467517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.467517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.468115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:22.468414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.471091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.479703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 59/119] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:22.484489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.486535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59abd7d8-9c4d-4da2-99d5-6fcb49b002c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.487203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abd7d8-9c4d-4da2-99d5-6fcb49b002c1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.487645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abd7d8-9c4d-4da2-99d5-6fcb49b002c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.487935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59abd7d8-9c4d-4da2-99d5-6fcb49b002c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.488559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.518825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.565849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abd7d8-9c4d-4da2-99d5-6fcb49b002c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.567366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59abd7d8-9c4d-4da2-99d5-6fcb49b002c1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.594957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1876e2f-918a-4b7d-849e-925c636673ac None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.635408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.636373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.636673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.636961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.638928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.640628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.647555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 60/120] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:22.647555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.676020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 61/121] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:22.676020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.676020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.676020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.694418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:22.695579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:22.728858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.752614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.761296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f3d34-6344-4c74-b1dd-8119bc84f4ea None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.771756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 62/122] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:22.780755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.797607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.798283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.798587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.798854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.799679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:22.799994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.801381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.810494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 61/123] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:22.811777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.902768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354712d6-0249-4997-b211-c96cb6ac48c1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.919242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f860a4-d49d-489c-a268-219b66d05491 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:22.938048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.938939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.939231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.939553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.940773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.942252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.944103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 63/124] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:22.944751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:22.958790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:22.959494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:22.959788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:22.960073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:22.960794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:22.962244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:22.965699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:22.994467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 62/125] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:48:22 2022] POST /identity/v3/projects => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:22.995328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.019608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:23.088084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b490dbd-33f4-4d34-b262-f3db8220f898 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.129213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776572-cad7-4230-9f57-e56156180801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.130105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776572-cad7-4230-9f57-e56156180801 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.130453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776572-cad7-4230-9f57-e56156180801 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.130674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25776572-cad7-4230-9f57-e56156180801 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.131956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776572-cad7-4230-9f57-e56156180801 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.133462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25776572-cad7-4230-9f57-e56156180801 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.135613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 64/126] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.136568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776572-cad7-4230-9f57-e56156180801 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.146081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1100592d-7f20-4dbd-890c-89e4f050a18c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.197134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.198209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.198499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.198782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.215241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:23.224447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:23.225478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 63/127] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:23.226263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.271329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25776572-cad7-4230-9f57-e56156180801 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.280992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:23.303645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:23.319885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.321327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.322338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.322825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.324325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.327384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.330511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 65/128] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.331802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.338481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d07430-33f0-4048-b1e2-f40809114c0a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.374725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.375782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.376138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.376477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.377776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.379633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.381188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 64/129] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.382025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.454272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de628c6-266c-4549-82e2-84da70dc6f59 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.491867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.492879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.493344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.493853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.494873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] RBAC: Adding query filter params (role.id=cb8bcd18ccd341c1b9653ba56d25ceea, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=7e17af398e2241a6b3774f42beb395cf) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:23.495390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.497046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.503476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 66/130] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/role_assignments?user.id=7e17af398e2241a6b3774f42beb395cf&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=cb8bcd18ccd341c1b9653ba56d25ceea => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.504545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.513210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4cb57-6d22-4d63-9737-81842a30a979 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.529504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:23.569668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:23.578495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.579194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.579564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.580301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.580950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:23.581337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.582791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.605307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 65/131] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.609717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.634010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9d24-069e-4fc0-9a49-5ee05bf02a91 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.690139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.691442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.691960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.692437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] PATH_INFO: `/v3/projects/f96f145b0e1b4199b98766672d83163d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.693846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] RBAC: Authorizing `identity:get_project(project_id=f96f145b0e1b4199b98766672d83163d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.695593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.697571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 67/132] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/projects/f96f145b0e1b4199b98766672d83163d => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.698574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.802764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2289696c-65e2-4c2b-9bfa-2dcfacfe979f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.806465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e069bd-d19d-4f25-ae32-6601eec78a06 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:23.832254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:23.845413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.846975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.847514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.847995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.849430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.851149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.853071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 68/133] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:23.854014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.896887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:23.897902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:23.898198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:23.898485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/ee104dedf13a4c7aa48858230c010f93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:23.904838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c253b10dbca4a15b11c1d888597ca3b, user_id=ee104dedf13a4c7aa48858230c010f93, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:23.907433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:23.934858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 66/134] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:23 2022] PUT /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/ee104dedf13a4c7aa48858230c010f93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:23.935622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:23.960768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c3fe1-87f3-4d3c-b298-0b659f371caf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:24.023653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:24.032118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:24.033423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:24.033948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:24.034426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] PATH_INFO: `/v3/projects/f96f145b0e1b4199b98766672d83163d/users/31594a42591343ab884a5efd018433c4/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:24.044282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] RBAC: Authorizing `identity:create_grant(project_id=f96f145b0e1b4199b98766672d83163d, user_id=31594a42591343ab884a5efd018433c4, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:24.046753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:24.069750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 69/135] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:23 2022] PUT /identity/v3/projects/f96f145b0e1b4199b98766672d83163d/users/31594a42591343ab884a5efd018433c4/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:24.140497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c59973-498e-46f8-b2ed-30f542902ab9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:24.196930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803986bf-ca98-41b4-af89-52f4fde1bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:24.197816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803986bf-ca98-41b4-af89-52f4fde1bb82 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:24.198110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803986bf-ca98-41b4-af89-52f4fde1bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:24.198396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803986bf-ca98-41b4-af89-52f4fde1bb82 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:24.204391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803986bf-ca98-41b4-af89-52f4fde1bb82 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:24.205905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803986bf-ca98-41b4-af89-52f4fde1bb82 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:24.207821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 67/136] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:23 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:24.213584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:24.307055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:24.365193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64d843-606c-41b3-9696-6d31fd903932 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:24.403328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfdaa85-837f-4bac-922a-a61748514b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:24.404261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdaa85-837f-4bac-922a-a61748514b50 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:24.404547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdaa85-837f-4bac-922a-a61748514b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:24.404878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfdaa85-837f-4bac-922a-a61748514b50 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/31594a42591343ab884a5efd018433c4/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:24.407300 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdaa85-837f-4bac-922a-a61748514b50 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c253b10dbca4a15b11c1d888597ca3b, user_id=31594a42591343ab884a5efd018433c4, role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:24.408785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfdaa85-837f-4bac-922a-a61748514b50 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:24.431643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 70/137] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:24 2022] PUT /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/31594a42591343ab884a5efd018433c4/roles/df14633e5004446c8c52b442c7d11a91 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:24.668057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c75ec-9f9b-421c-8eeb-863cfc05b2e8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:24.668399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c75ec-9f9b-421c-8eeb-863cfc05b2e8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:24.668647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905c75ec-9f9b-421c-8eeb-863cfc05b2e8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:24.670078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 68/138] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:24.676239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:24.676534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:24.676818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:24.697736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:24.698943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:24.782010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 71/139] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:24.813654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c75ec-9f9b-421c-8eeb-863cfc05b2e8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:24.953228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905c75ec-9f9b-421c-8eeb-863cfc05b2e8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:24.955673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:25.000037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:25.000872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:25.001167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:25.001449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:25.016093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:25.024169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:25.025013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:25.025767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 69/140] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:25.031666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:25.088210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:25.159711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0330c9b-665d-4582-8164-3f2f3e05d69a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:25.197027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:25.197707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:25.197996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:25.198278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:25.199034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:25.199372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:25.200767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:25.210263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 72/141] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:25.215690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:25.376370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a9f5a8-afbe-42a4-b4f0-49f14bb3c88a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:25.421461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:25.422353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:25.422646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:25.422933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:25.424178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:25.425638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:25.427311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 70/142] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:25 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:25.432950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:25.554212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ff129-f4ff-436f-8faa-283a23a6c673 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:25.556127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:25.591264 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:25.591987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:25.592860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:25.593180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:25.664597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:25.666111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:25.767541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 73/143] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:25.773125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:25.805299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:25.875024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898611bc-bbc4-426b-a1a9-88eca95e1d4f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:25.909129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeacbf3c-ce05-41a0-9c28-1a2fbe469bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:25.910044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbf3c-ce05-41a0-9c28-1a2fbe469bcf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:25.910332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbf3c-ce05-41a0-9c28-1a2fbe469bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:25.910611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeacbf3c-ce05-41a0-9c28-1a2fbe469bcf None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:25.911831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacbf3c-ce05-41a0-9c28-1a2fbe469bcf None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:25.913271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeacbf3c-ce05-41a0-9c28-1a2fbe469bcf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:25.915128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 71/144] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:25 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:25.921289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:25.931599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:26.021755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68406a9f-2c7e-48d7-8a16-69923ae18258 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:26.056512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d6ae7-bcd3-40ce-a7da-cf70f2559d60 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.057096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d6ae7-bcd3-40ce-a7da-cf70f2559d60 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.057520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:26.058150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d6ae7-bcd3-40ce-a7da-cf70f2559d60 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.058248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.058469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.058755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.059629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=31594a42591343ab884a5efd018433c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:26.059975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:26.061188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 72/145] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:26.061497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:26.069272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.069561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.069838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.075323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 74/146] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:25 2022] GET /identity/v3/role_assignments?user.id=31594a42591343ab884a5efd018433c4&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:26.089578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:26.090807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:26.168900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 73/147] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:26.193467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:26.270394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:26.311752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c90c2d-7f83-4186-b672-6826407d0786 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:26.347071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202d394-d239-4438-9f75-e8f9c10ea3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:26.347906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202d394-d239-4438-9f75-e8f9c10ea3db None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.348190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202d394-d239-4438-9f75-e8f9c10ea3db None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.348468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202d394-d239-4438-9f75-e8f9c10ea3db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.349657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202d394-d239-4438-9f75-e8f9c10ea3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:26.351102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202d394-d239-4438-9f75-e8f9c10ea3db None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:26.352795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 75/148] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:26.358405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:26.390255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:26.464714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae253601-e4d8-4ff8-9947-a627411cebfc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:26.500272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bdcf9-d22a-4b16-9567-7ec3535028ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:26.500941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bdcf9-d22a-4b16-9567-7ec3535028ff None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.501227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bdcf9-d22a-4b16-9567-7ec3535028ff None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.501498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bdcf9-d22a-4b16-9567-7ec3535028ff None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.502222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bdcf9-d22a-4b16-9567-7ec3535028ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:26.503816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bdcf9-d22a-4b16-9567-7ec3535028ff None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:26.526857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 74/149] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:26 2022] POST /identity/v3/users => generated 283 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:26.966294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa6197-460b-42ac-a0ca-a4dbd71e7abf None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.966666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa6197-460b-42ac-a0ca-a4dbd71e7abf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.966853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fa6197-460b-42ac-a0ca-a4dbd71e7abf None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.968307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 76/150] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:26.974579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.975041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.975131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.981053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa168c-e641-4285-b3c0-72273d24b3e4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.981618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa168c-e641-4285-b3c0-72273d24b3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.982096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa168c-e641-4285-b3c0-72273d24b3e4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.984405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 77/151] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:26.989792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:26.990349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:26.990853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:26.994237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:26.995718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:27.010349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:27.012009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:27.079721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 75/152] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:27.089715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 78/153] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:27.096220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.115086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:27.117573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.185084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:27.213892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75467e-e64b-4ff6-a118-26b0dd35adc4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.226321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc566e7f-5b9a-4096-8215-5c2e14126912 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.258713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0af44-f2d0-43a7-8ee5-89e8aebb8fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.259569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0af44-f2d0-43a7-8ee5-89e8aebb8fcc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.259895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0af44-f2d0-43a7-8ee5-89e8aebb8fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.260129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0af44-f2d0-43a7-8ee5-89e8aebb8fcc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.264295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.265386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.265886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.266375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.272452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e0af44-f2d0-43a7-8ee5-89e8aebb8fcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:27.278373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-87e0af44-f2d0-43a7-8ee5-89e8aebb8fcc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:27.281249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:27.282153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:27.282956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:27.288517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 76/154] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:27.288633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:27.289415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:27.290642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:27.291730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1402926-6d33-4e00-acb3-e40522cc4538 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.291730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1402926-6d33-4e00-acb3-e40522cc4538 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.292011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 79/155] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:27.293126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1402926-6d33-4e00-acb3-e40522cc4538 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.293320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.293483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 77/156] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:27.297157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.297451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.297736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.318881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:27.320298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:27.408048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 78/157] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:27.408798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.419645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:27.429158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553bba-f82d-42f0-95db-51fe1d5f6705 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.457561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:27.474471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.475532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.475818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.476093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.477022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:27.477644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:27.479421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9894fa0-2a53-49c3-a5d6-5d9fff4e2420 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:27.490090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 80/158] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:27.491476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44f20-8d4e-4852-94bd-c3b8b437d3e6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.491770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44f20-8d4e-4852-94bd-c3b8b437d3e6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.492044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be44f20-8d4e-4852-94bd-c3b8b437d3e6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.493425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 81/159] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:27.493963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44f20-8d4e-4852-94bd-c3b8b437d3e6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.531475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8739a-f474-4ca5-a7fc-ddedd115fd04 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.581229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.583703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.584232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.584712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.585748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:27.586262 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:27.587907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:27.603331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 79/160] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:27.604563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.624882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be44f20-8d4e-4852-94bd-c3b8b437d3e6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.662023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5121d185-9904-4ffe-9878-c78802feb067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.662770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121d185-9904-4ffe-9878-c78802feb067 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.663055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121d185-9904-4ffe-9878-c78802feb067 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.663360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5121d185-9904-4ffe-9878-c78802feb067 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.678522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5121d185-9904-4ffe-9878-c78802feb067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5121d185-9904-4ffe-9878-c78802feb067 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:27.692101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:27.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:27.693809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:27.696334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 82/161] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:27.697932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.698435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.699130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:27.699871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.719970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:27.722030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:27.755705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c443a0b-13ef-49bd-ba21-d7cba9dda17d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.768961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:27.797325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 83/162] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:27.798755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.831913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.832870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.833157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.833486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] PATH_INFO: `/v3/users/7e17af398e2241a6b3774f42beb395cf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.834785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] RBAC: Authorizing `identity:get_user(user_id=7e17af398e2241a6b3774f42beb395cf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:27.836311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:27.838332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 80/163] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/users/7e17af398e2241a6b3774f42beb395cf => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:27.839283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:27.910607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daae9444-a1e0-4442-b78c-1f7adaa7c759 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:27.948124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:27.949445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:27.949942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:27.950447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:27.952003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:27.953732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:27.956028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 84/164] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:27.957217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.008388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:28.027524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8668354-d2fd-49c0-b635-bd1c31d11163 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:28.105900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:28.106584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:28.106865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:28.107143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:28.107991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:28.108301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:28.109685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:28.115182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:28.118286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 81/165] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:28.120587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.133213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee690a7-fe19-45fc-ab94-bde75df95837 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:28.227988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cabe17-5898-4902-8dd2-e19eeaf3311e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:28.261125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:28.262383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:28.262662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:28.262936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:28.266501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:28.267433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:28.267725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:28.267996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:28.269346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:28.271830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:28.273616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 82/166] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:28 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:28.274420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.286284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:28.294195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:28.295052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:28.296427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:28.307963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 85/167] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:28.308745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:28.379173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cadef3b-12b1-4482-a91d-e6afb3f27ada None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:28.422140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:28.422790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:28.423072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:28.423376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:28.523472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb932f95-8af5-4a2f-a8ef-de0ccd33ad55 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:28.526488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:28.556736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:28.558222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:28.565612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:28.566769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:28.567279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:28.567773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] PATH_INFO: `/v3/users/31594a42591343ab884a5efd018433c4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:28.570201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] RBAC: Authorizing `identity:get_user(user_id=31594a42591343ab884a5efd018433c4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:28.571929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:28.574040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 86/168] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:28 2022] GET /identity/v3/users/31594a42591343ab884a5efd018433c4 => generated 275 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:28.574992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.695665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:28.726507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06511fa8-3b21-4ffb-9558-7e8fcd941062 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:28.757959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 83/169] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:28.758645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.793950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:28.794761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:28.795041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:28.795916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/7e17af398e2241a6b3774f42beb395cf/roles/cb8bcd18ccd341c1b9653ba56d25ceea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:28.799138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] RBAC: Authorizing `identity:create_grant(project_id=6c253b10dbca4a15b11c1d888597ca3b, user_id=7e17af398e2241a6b3774f42beb395cf, role_id=cb8bcd18ccd341c1b9653ba56d25ceea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:28.800637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:28.823193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 87/170] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:28 2022] PUT /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/users/7e17af398e2241a6b3774f42beb395cf/roles/cb8bcd18ccd341c1b9653ba56d25ceea => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:28.823923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:28.887430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:28.936361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac39853-a2ee-4f84-981a-a7d628cbbc23 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.018893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae69d5-9851-4049-a5b4-9785a1f872ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.021642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.022820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.023395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.023825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.024723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:29.025170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.026847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.050356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 84/171] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:29.051883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:29.084671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.085350 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.085647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.085932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.111635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:29.186461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.187978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.208939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe580e3c-5b76-4024-8889-bdddfee5e9a9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.220272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:29.252861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.254049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.254494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.254957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] PATH_INFO: `/v3/projects/f96f145b0e1b4199b98766672d83163d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.256306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] RBAC: Authorizing `identity:get_project(project_id=f96f145b0e1b4199b98766672d83163d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.258102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.260093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 85/172] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:29 2022] GET /identity/v3/projects/f96f145b0e1b4199b98766672d83163d => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:29.260976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:29.299708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 88/173] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:29.300454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:29.401278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99a802-c58c-4367-b125-8f7b7a3059e3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.411621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:29.421101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a74374-cc46-471b-a33c-80a8621df31a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.431644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:29.443664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.444364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.444663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.444951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.445931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=f96f145b0e1b4199b98766672d83163d, user.id=31594a42591343ab884a5efd018433c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:29.446499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.448014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.454636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 89/174] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:29 2022] GET /identity/v3/role_assignments?user.id=31594a42591343ab884a5efd018433c4&scope.project.id=f96f145b0e1b4199b98766672d83163d&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:29.460588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:29.512001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.512920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.513213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.513498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] PATH_INFO: `/v3/users/ee104dedf13a4c7aa48858230c010f93` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.520138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] RBAC: Authorizing `identity:get_user(user_id=ee104dedf13a4c7aa48858230c010f93)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.521666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.523538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 86/175] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:29 2022] GET /identity/v3/users/ee104dedf13a4c7aa48858230c010f93 => generated 247 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:29.532178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:29.666375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28470bc-41bc-4b19-ae33-59c17fd1f7b7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.695687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1328b50-4784-40a3-8666-d57060492f94 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:29.707800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a7974-51c8-4f64-aaa0-663c18cf23ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.708528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a7974-51c8-4f64-aaa0-663c18cf23ff None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.708863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a7974-51c8-4f64-aaa0-663c18cf23ff None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.709149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a7974-51c8-4f64-aaa0-663c18cf23ff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.749435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:29.782521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be640ae3-9c68-4d11-a211-d8f76ed99b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:29.783368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640ae3-9c68-4d11-a211-d8f76ed99b62 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.783653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640ae3-9c68-4d11-a211-d8f76ed99b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.783932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be640ae3-9c68-4d11-a211-d8f76ed99b62 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.788018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be640ae3-9c68-4d11-a211-d8f76ed99b62 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.789574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be640ae3-9c68-4d11-a211-d8f76ed99b62 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.791455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 90/176] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:29 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:29.792569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464235-7326-4710-920e-442ee22fcd9f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:29.792857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464235-7326-4710-920e-442ee22fcd9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:29.793131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1464235-7326-4710-920e-442ee22fcd9f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:29.794367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 91/177] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:29.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1464235-7326-4710-920e-442ee22fcd9f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:29.800153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a7974-51c8-4f64-aaa0-663c18cf23ff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:29.803705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a7974-51c8-4f64-aaa0-663c18cf23ff None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:29.916359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:29.990102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1464235-7326-4710-920e-442ee22fcd9f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.012353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 87/178] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:30.018708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.019007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.019304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.056767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:30.057978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:30.062853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.063545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.063831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.064106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.064938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] RBAC: Adding query filter params (role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=31594a42591343ab884a5efd018433c4) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:30.065250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:30.066706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:30.085508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 92/179] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:29 2022] GET /identity/v3/role_assignments?user.id=31594a42591343ab884a5efd018433c4&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=df14633e5004446c8c52b442c7d11a91 => generated 608 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:30.088232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.182588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:30.198680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 88/180] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:30.246612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.317451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82777581-611f-4666-9a49-bf1f2bb230b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.362608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:30.381983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f5099-2662-49d8-a6de-0ca7d333381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.382889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f5099-2662-49d8-a6de-0ca7d333381f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.383171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f5099-2662-49d8-a6de-0ca7d333381f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.383479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f5099-2662-49d8-a6de-0ca7d333381f None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.386353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f5099-2662-49d8-a6de-0ca7d333381f None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:30.387938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f5099-2662-49d8-a6de-0ca7d333381f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:30.391193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 93/181] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:30.398525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a6c6-bffe-4279-8dd6-63c0fb471a00 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.398814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a6c6-bffe-4279-8dd6-63c0fb471a00 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.399090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2a6c6-bffe-4279-8dd6-63c0fb471a00 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.400540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 94/182] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:30.403989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a6c6-bffe-4279-8dd6-63c0fb471a00 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.436626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685ac6c9-d96c-43c3-9a8d-c5871b4aca14 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.484353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf16f6f-1b82-432f-b320-efcb40b3395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.485445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16f6f-1b82-432f-b320-efcb40b3395b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.485927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16f6f-1b82-432f-b320-efcb40b3395b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.486383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf16f6f-1b82-432f-b320-efcb40b3395b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.497158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf16f6f-1b82-432f-b320-efcb40b3395b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6cf16f6f-1b82-432f-b320-efcb40b3395b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:30.504885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:30.505757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:30.506542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 89/183] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:30.507539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38b902-b853-4e27-a863-57d6893ba5dc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.507793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38b902-b853-4e27-a863-57d6893ba5dc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.508047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f38b902-b853-4e27-a863-57d6893ba5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.532948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3f38b902-b853-4e27-a863-57d6893ba5dc None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:30.534150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3f38b902-b853-4e27-a863-57d6893ba5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:30.551497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2a6c6-bffe-4279-8dd6-63c0fb471a00 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.587698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.588750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:30.589250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.590342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.590843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.592539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=ee104dedf13a4c7aa48858230c010f93) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:30.593368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:30.595031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc05f0-74fc-4409-9979-7f7489e6e7df None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:30.601370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 95/184] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/role_assignments?user.id=ee104dedf13a4c7aa48858230c010f93&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:30.602854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519afa8-2a8d-4b51-b25d-a164104a38a4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.603208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519afa8-2a8d-4b51-b25d-a164104a38a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.603599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9519afa8-2a8d-4b51-b25d-a164104a38a4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.607306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 96/185] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:30.607306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519afa8-2a8d-4b51-b25d-a164104a38a4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.615611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 90/186] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:30.617101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.617504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.617890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.650620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:30.652185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:30.690934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:30.737277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 91/187] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:30.738015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.751817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9519afa8-2a8d-4b51-b25d-a164104a38a4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.791059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.792111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.792543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.792965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.793921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:30.794423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:30.796056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:30.806349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 97/188] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:30.807281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.863039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d5fd86-770d-4d5c-8425-498886d7e920 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.881328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:30.906638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.907481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.907767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.908045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.921557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:30.930100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:30.930981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:30.931812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 92/189] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:30.932459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:30.938461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a83f356-fe4e-4ff7-a874-1cf914606e89 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:30.962187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:30.976484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:30.977291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:30.977579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:30.977864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:30.979059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:30.980561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:30.982201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 98/190] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:30.987277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.039975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f04da6-6f9a-44c0-942f-1058ebf33d86 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.073093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.073977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.074267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.074540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] PATH_INFO: `/v3/users/cd63cffde1104f34be0683986641d472` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.088081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] RBAC: Authorizing `identity:get_user(user_id=cd63cffde1104f34be0683986641d472)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.089529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.091219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 93/191] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/users/cd63cffde1104f34be0683986641d472 => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:31.092522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.095121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208d160a-c21c-4bc5-9525-791f7ce97348 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.123166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.134649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.135357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.135648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.135923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.136669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:31.137002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.138392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.150778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 99/192] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:31.151550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.199506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd2d689-abd8-421f-9773-06a6ecde10ab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.206485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.234948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.235659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.235943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.236220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.236934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.238381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.258418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9d2a42-6bb7-47e7-a454-58c042d6dcc8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.267521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 94/193] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:31 2022] POST /identity/v3/users => generated 297 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:31.267521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.315748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab948c8-0daa-4211-ad3a-c9a8583b472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.316497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab948c8-0daa-4211-ad3a-c9a8583b472e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.316856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab948c8-0daa-4211-ad3a-c9a8583b472e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.317856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab948c8-0daa-4211-ad3a-c9a8583b472e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.385384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.402943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af267241-c171-4bbc-a46a-2bae69af8fa1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.414243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab948c8-0daa-4211-ad3a-c9a8583b472e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.415787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab948c8-0daa-4211-ad3a-c9a8583b472e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.443744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86585c67-0592-4812-be82-6941fc76ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.444626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86585c67-0592-4812-be82-6941fc76ff3c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.444904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86585c67-0592-4812-be82-6941fc76ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.445177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86585c67-0592-4812-be82-6941fc76ff3c None admin] PATH_INFO: `/v3/users/cd63cffde1104f34be0683986641d472` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.446514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86585c67-0592-4812-be82-6941fc76ff3c None admin] RBAC: Authorizing `identity:get_user(user_id=cd63cffde1104f34be0683986641d472)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.448067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86585c67-0592-4812-be82-6941fc76ff3c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.450514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 95/194] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:31 2022] GET /identity/v3/users/cd63cffde1104f34be0683986641d472 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:31.451111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da37b0-b3d7-4b4c-b370-03e74ff1b1c5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.451989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da37b0-b3d7-4b4c-b370-03e74ff1b1c5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.452266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da37b0-b3d7-4b4c-b370-03e74ff1b1c5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.453624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 96/195] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:31.458116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da37b0-b3d7-4b4c-b370-03e74ff1b1c5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.476377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.532692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 100/196] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:31.534026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.534318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.534603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.558755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:31.560015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:31.586391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da37b0-b3d7-4b4c-b370-03e74ff1b1c5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.630631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.631319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.631586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.631968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 101/197] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:31.632079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.632637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.675002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.723762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.725350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.736753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.756832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820d615-cf07-4c4f-8e7b-27ddce714400 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.794002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c82482-8285-4254-a758-653da083354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.795714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c82482-8285-4254-a758-653da083354a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.796072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c82482-8285-4254-a758-653da083354a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.796414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c82482-8285-4254-a758-653da083354a None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.797702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c82482-8285-4254-a758-653da083354a None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.799387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c82482-8285-4254-a758-653da083354a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:31.801368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 102/198] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:31 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:31.802195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c82482-8285-4254-a758-653da083354a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.837850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 97/199] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:31.838420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:31.901969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:31.924823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c82482-8285-4254-a758-653da083354a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.951858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c0fc18-39e4-47f6-a116-d569b01e561d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:31.978598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.980122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.980410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.980953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.990473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:31.991179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:31.991496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:31.991776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:31.992616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=cd63cffde1104f34be0683986641d472) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:31.992929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:31.994044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:31.994422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:32.001037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 98/200] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:31 2022] GET /identity/v3/role_assignments?user.id=cd63cffde1104f34be0683986641d472&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:32.001829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:32.004683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:32.005562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:32.006819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 103/201] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:32.012485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:32.042808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:32.195696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e410e6c-ca9d-4300-a3a2-444757b6a9a4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:32.230685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:32.246838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:32.247968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:32.248351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:32.248700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:32.249383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236da77-8437-41bd-aaf5-50ea687644d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:32.250045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:32.251858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:32.253660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 99/202] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:32 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:32.260254 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:32.348500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:32.349179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:32.349464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:32.349741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:32.350489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:32.350796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:32.352216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:32.366953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 104/203] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:32.372728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:32.396610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e439547-4536-43dd-8fdc-108c6b08ad82 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:32.458174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:32.458747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:32.459032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:32.459326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:32.460168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] RBAC: Adding query filter params (role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=cd63cffde1104f34be0683986641d472) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:32.460480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:32.461913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:32.470283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 100/204] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:32 2022] GET /identity/v3/role_assignments?user.id=cd63cffde1104f34be0683986641d472&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=df14633e5004446c8c52b442c7d11a91 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:32.497166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:32.547265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d957c-eaaf-4f19-90b3-4d68360dd90a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:32.589532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:32.590416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:32.590697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:32.590970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] PATH_INFO: `/v3/users/7e17af398e2241a6b3774f42beb395cf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:32.592790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] RBAC: Authorizing `identity:get_user(user_id=7e17af398e2241a6b3774f42beb395cf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:32.594320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:32.596048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 105/205] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:32 2022] GET /identity/v3/users/7e17af398e2241a6b3774f42beb395cf => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:32.604339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:32.682413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:32.733754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a73dc9-1d05-4db6-947d-b0d725bf84fb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:32.776352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:32.777009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:32.777292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:32.777568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:32.860564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:32.862306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:32.925522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:32.976333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 101/206] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:32.984450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:33.099695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89110c13-2da7-4347-ace1-f9fe9b865e49 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:33.133985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ffe97-70de-43c8-9a9d-42360f24e076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:33.135107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ffe97-70de-43c8-9a9d-42360f24e076 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.135551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ffe97-70de-43c8-9a9d-42360f24e076 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.135961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ffe97-70de-43c8-9a9d-42360f24e076 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:33.137288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ffe97-70de-43c8-9a9d-42360f24e076 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:33.138921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ffe97-70de-43c8-9a9d-42360f24e076 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:33.141072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 106/207] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:32 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:33.148324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:33.280360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7594611-ca95-4336-b766-84270fd27f1a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:33.318004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:33.318943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.319418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.319834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:33.320804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] RBAC: Adding query filter params (role.id=cb8bcd18ccd341c1b9653ba56d25ceea, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b, user.id=7e17af398e2241a6b3774f42beb395cf) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:33.321253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:33.322836 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbeebcd-35bc-4290-8c7e-4462382bdd2d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:33.329430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 102/208] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:33 2022] GET /identity/v3/role_assignments?user.id=7e17af398e2241a6b3774f42beb395cf&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=cb8bcd18ccd341c1b9653ba56d25ceea => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:33.692486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717ed05-3b2d-468e-8a85-88c9911f1269 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.693493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717ed05-3b2d-468e-8a85-88c9911f1269 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.693980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717ed05-3b2d-468e-8a85-88c9911f1269 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:33.696022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 107/209] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:33.701096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30c7b-a06a-4e86-84e1-e9f2e3d54af5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.701646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30c7b-a06a-4e86-84e1-e9f2e3d54af5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.702115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec30c7b-a06a-4e86-84e1-e9f2e3d54af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:33.719278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3ec30c7b-a06a-4e86-84e1-e9f2e3d54af5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:33.720792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3ec30c7b-a06a-4e86-84e1-e9f2e3d54af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:33.732943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:33.790359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 103/210] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:33.816990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717ed05-3b2d-468e-8a85-88c9911f1269 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:33.829354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:33.920066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6cde5-33a4-4cbd-b5fa-001f9d846288 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.920739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6cde5-33a4-4cbd-b5fa-001f9d846288 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.921247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6cde5-33a4-4cbd-b5fa-001f9d846288 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:33.922864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 104/211] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:33.928313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.928847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.929353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:33.951111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:33.952909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:33.956614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717ed05-3b2d-468e-8a85-88c9911f1269 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:33.995133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:33.996357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:33.996915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:33.997396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.015885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:34.025252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:34.026138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:34.026959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:34.029955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 108/212] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:34.035411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.043301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 105/213] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:34.063160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.125688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.151700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.167833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc86513a-c54e-4887-9e5c-52bbe44fe902 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.176522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e892b288-b0cb-4f2d-9fd7-4a4e7e5189ea None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.207479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.208534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.208976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.209392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.210407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:34.210856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.212640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.213436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.214402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.214833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.215274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.216643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.218251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.220097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 106/214] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:34.223314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 109/215] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:34.226875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.227116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.328281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1a36f-7dd9-4d95-bf20-b06b24cff5e7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.339054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d91b04-0612-4c2d-80c1-ff6a9bf12ce0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.363774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5443b75-f2ee-410d-aee8-28f95acaf50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.364984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443b75-f2ee-410d-aee8-28f95acaf50f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.365594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443b75-f2ee-410d-aee8-28f95acaf50f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.365949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5443b75-f2ee-410d-aee8-28f95acaf50f None admin] PATH_INFO: `/v3/users/3ac67b1e20a84324a3ae8a037d1287ac` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.367771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.374456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c5eda2-6311-4d57-b9d3-0bb5e06bdfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.375118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5eda2-6311-4d57-b9d3-0bb5e06bdfbf None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.375437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5eda2-6311-4d57-b9d3-0bb5e06bdfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.375714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c5eda2-6311-4d57-b9d3-0bb5e06bdfbf None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.376417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5eda2-6311-4d57-b9d3-0bb5e06bdfbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.377855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c5eda2-6311-4d57-b9d3-0bb5e06bdfbf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.380715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5443b75-f2ee-410d-aee8-28f95acaf50f None admin] RBAC: Authorizing `identity:get_user(user_id=3ac67b1e20a84324a3ae8a037d1287ac)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.383450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5443b75-f2ee-410d-aee8-28f95acaf50f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.388924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 107/216] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/users/3ac67b1e20a84324a3ae8a037d1287ac => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:34.391021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485d611-e542-452a-9ade-bbe5f89b033b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.391668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485d611-e542-452a-9ade-bbe5f89b033b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.391987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485d611-e542-452a-9ade-bbe5f89b033b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.393482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 108/217] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:34.396268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485d611-e542-452a-9ade-bbe5f89b033b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.399733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.404934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 110/218] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:34 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:34.406040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.406335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.406831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.438487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:34.441146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:34.523091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485d611-e542-452a-9ade-bbe5f89b033b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.540254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 111/219] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:34.564562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.565675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.566834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.567319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.568680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.649069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.650551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.654814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.680543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fecc58-edf6-4572-8d2e-12165f16731f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.699763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.728950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.729761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.730050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.730494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.742978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:34.754730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:34.755714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:34.757281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 112/220] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:34.762221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.776310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 109/221] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:34.783342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.899700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f3ffb-f2fd-44b0-90b2-50898fcd495a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.926081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056ac81e-e65a-4ccf-abc8-dac9d6a759b8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:34.928776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:34.952436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.953114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.953398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.953675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.954439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:34.954775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.956176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.972511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:34.975678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:34.976169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:34.976646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:34.977093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 113/222] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:34.978573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:34.980283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:34.981359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:34.984206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 110/223] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:34.993834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:35.068195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:35.174121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e426034e-7390-445d-acc5-42515c06dbf5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:35.174762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4802a69-3c80-4d45-900a-5b703dee884e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:35.216897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:35.217604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.217838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.218113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.219066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=3ac67b1e20a84324a3ae8a037d1287ac) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:35.219421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:35.219820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e13f04-942a-4ee9-801c-240b6953e175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:35.220853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f33f37-16e1-4575-80c4-c6766b819731 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:35.221421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13f04-942a-4ee9-801c-240b6953e175 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.221908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13f04-942a-4ee9-801c-240b6953e175 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.222379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e13f04-942a-4ee9-801c-240b6953e175 None admin] PATH_INFO: `/v3/users/cd63cffde1104f34be0683986641d472` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.223837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e13f04-942a-4ee9-801c-240b6953e175 None admin] RBAC: Authorizing `identity:get_user(user_id=cd63cffde1104f34be0683986641d472)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:35.225504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e13f04-942a-4ee9-801c-240b6953e175 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:35.227480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 111/224] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/role_assignments?user.id=3ac67b1e20a84324a3ae8a037d1287ac&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:35.227826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 114/225] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:34 2022] GET /identity/v3/users/cd63cffde1104f34be0683986641d472 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:35.228831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e5d2-d4f6-4299-ab6c-d3590de6f1be None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.229119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e5d2-d4f6-4299-ab6c-d3590de6f1be None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.229401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e5e5d2-d4f6-4299-ab6c-d3590de6f1be None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.231292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 112/226] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:35.237023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.243323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.243323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.244687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e5d2-d4f6-4299-ab6c-d3590de6f1be None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:35.284470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:35.285987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:35.291427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:35.376443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e5e5d2-d4f6-4299-ab6c-d3590de6f1be None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:35.441827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:35.442696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.442979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.443272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.445034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:35.446495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:35.449691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 113/227] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:35 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:35.458324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:35.465613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 115/228] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:35.495415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:35.573401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bdbed3-092f-4068-9ef2-93a3374aa7e2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:35.575003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:35.575767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:35.611780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:35.612679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.612964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.613238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/cd63cffde1104f34be0683986641d472/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.615405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] RBAC: Authorizing `identity:create_grant(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5, user_id=cd63cffde1104f34be0683986641d472, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:35.616856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:35.635242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 114/229] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:35 2022] PUT /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/cd63cffde1104f34be0683986641d472/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:35.716325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3fe8d1-958a-4549-9805-a73e820a04c1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:35.761182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:35.762007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.762296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.762575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.780542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:35.788948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:35.791031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:35.793102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 116/230] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:35.799395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:35.905706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533517d-af62-4075-a755-6d078d38ce0a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:35.942575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:35.943268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:35.943568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:35.943850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:35.944613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:35.944922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:35.946316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:35.956347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 115/231] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:35.962238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:36.069412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:36.075677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ef0621-a6a1-4da1-9fb3-8619d30075eb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:36.120100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:36.120986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:36.121271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:36.121550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] PATH_INFO: `/v3/users/cd63cffde1104f34be0683986641d472` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:36.123396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] RBAC: Authorizing `identity:get_user(user_id=cd63cffde1104f34be0683986641d472)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:36.124873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:36.126678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 117/232] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:35 2022] GET /identity/v3/users/cd63cffde1104f34be0683986641d472 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:36.132109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:36.159224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:36.237241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5830ab2e-5a9d-499f-bace-859210fc526b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:36.270338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab6e83-5c26-42db-ae29-8a066a28c483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:36.271202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab6e83-5c26-42db-ae29-8a066a28c483 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:36.271519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab6e83-5c26-42db-ae29-8a066a28c483 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:36.271797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab6e83-5c26-42db-ae29-8a066a28c483 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:36.272992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab6e83-5c26-42db-ae29-8a066a28c483 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:36.274435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab6e83-5c26-42db-ae29-8a066a28c483 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:36.281461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 116/233] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:36 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:36.282839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:36.403462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d4e83-9dfa-4d13-a2a2-bd0cb351ea27 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:36.447456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171e0385-9ff0-4244-a563-8acb00cfdbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:36.448346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e0385-9ff0-4244-a563-8acb00cfdbe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:36.448627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e0385-9ff0-4244-a563-8acb00cfdbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:36.448901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171e0385-9ff0-4244-a563-8acb00cfdbe4 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/cd63cffde1104f34be0683986641d472/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:36.457145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e0385-9ff0-4244-a563-8acb00cfdbe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5, user_id=cd63cffde1104f34be0683986641d472, role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:36.458622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171e0385-9ff0-4244-a563-8acb00cfdbe4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:36.477131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 118/234] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:36 2022] PUT /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/cd63cffde1104f34be0683986641d472/roles/df14633e5004446c8c52b442c7d11a91 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:37.096672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab0995-81cb-4b04-a8a9-a6fd0f0e4025 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.097171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab0995-81cb-4b04-a8a9-a6fd0f0e4025 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.097171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ab0995-81cb-4b04-a8a9-a6fd0f0e4025 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.098889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 117/235] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:37.103945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.104349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.104518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.128346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:37.129583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:37.154572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:37.176525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d43d96-70e0-4f00-b729-68d0e01096d3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.177147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d43d96-70e0-4f00-b729-68d0e01096d3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.177651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d43d96-70e0-4f00-b729-68d0e01096d3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.179478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 118/236] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:37.196810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.197400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.197968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.208026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 119/237] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:37.233423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:37.252290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:37.253609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:37.370363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a8c761-e0bd-416e-88fb-746501c87cdd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:37.373641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 119/238] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:37.400330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:37.424928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:37.445627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:37.448097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.451985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.452475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.453885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:37.455690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:37.457616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 120/239] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:37.467217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:37.495390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:37.571886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5d7b4-4b66-4351-b90d-34536efb0883 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:37.614251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fcc86-4ded-4ddf-9644-72e7e8e4251f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:37.618710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:37.619559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.619848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.620126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.640115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:37.653508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:37.655008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:37.655799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 120/240] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:37.660928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203e8461-9298-489e-8626-7ef8d64fe7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:37.661343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:37.662069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8461-9298-489e-8626-7ef8d64fe7fe None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.662548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8461-9298-489e-8626-7ef8d64fe7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.663033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203e8461-9298-489e-8626-7ef8d64fe7fe None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.663977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8461-9298-489e-8626-7ef8d64fe7fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:37.665628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203e8461-9298-489e-8626-7ef8d64fe7fe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:37.695353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 121/241] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:37 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:37.759639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef93565-2ae5-4175-9cca-278184151ce6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.760301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef93565-2ae5-4175-9cca-278184151ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.760786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef93565-2ae5-4175-9cca-278184151ce6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.762740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 122/242] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:37.778765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.779456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.779808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.801361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:37.808385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:37.811925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:37.835917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caeef3d-c914-43c4-b76c-92d1776a49ac None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:37.909170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:37.909769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:37.910052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:37.910324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:37.912212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:37.912538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:37.914886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:37.933226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 121/243] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:37.940305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:37.940738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 123/244] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:37.967945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:37.981481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.074101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70459d1-b474-4949-90d9-b264447dbb68 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.105891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e663b-83fd-40cf-afee-0a149b4a5358 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.118654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.119675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.122323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.122805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] PATH_INFO: `/v3/users/1627ceb579da41cc8b73d251ccd21999` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.139914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] RBAC: Authorizing `identity:get_user(user_id=1627ceb579da41cc8b73d251ccd21999)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:38.141482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:38.143359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 122/245] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:37 2022] GET /identity/v3/users/1627ceb579da41cc8b73d251ccd21999 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:38.149085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:38.150666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfced42-f710-42eb-a426-3a4825cacf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.151506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfced42-f710-42eb-a426-3a4825cacf27 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.152431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.153042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfced42-f710-42eb-a426-3a4825cacf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.153228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfced42-f710-42eb-a426-3a4825cacf27 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.168728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfced42-f710-42eb-a426-3a4825cacf27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8bfced42-f710-42eb-a426-3a4825cacf27 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:38.177597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:38.178670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:38.179623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 124/246] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:38.180282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9238c-9051-45f1-9283-531681a0e80c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.180354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9238c-9051-45f1-9283-531681a0e80c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.180773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be9238c-9051-45f1-9283-531681a0e80c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.181884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 125/247] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:38.187119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.187443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.187727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.205798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:38.207038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:38.274567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c535e3-d965-4a0a-86fe-4e4cb8aea54a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.279535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.285724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 126/248] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:38.286591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:38.317134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.317897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.318189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.318467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.396030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810dd3f-270b-4b0a-acaa-8d17bb4758f4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.401086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:38.402558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:38.404748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.451509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.452578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.452983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.453270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.454344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:38.457629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:38.460291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:38.483377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 127/249] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:38.488488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:38.559944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.590751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 123/250] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 05:48:38.592119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:38.601255 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253539d-d029-4db0-9fdd-2c4d1cd56e39 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.647978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.649178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.649677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.650158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.692188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:38.696559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5bbeee-b112-4e95-ae17-5279e0142b78 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.707686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:38.708141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:38.709207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:38.710033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 128/251] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:38.710668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:38.734102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.735042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.735361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.735682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] PATH_INFO: `/v3/users/3ac67b1e20a84324a3ae8a037d1287ac` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.736920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] RBAC: Authorizing `identity:get_user(user_id=3ac67b1e20a84324a3ae8a037d1287ac)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:38.738513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:38.740392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 124/252] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/users/3ac67b1e20a84324a3ae8a037d1287ac => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:38.741167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:38.909779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2fa8c4-f560-46ea-a3e7-88de354564fa None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.930050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adf598-108f-420f-ac6d-43ac0bca1814 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:38.953291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:38.976736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:38.978953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:38.980232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:38.980975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:38.984166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:38.985858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:38.992345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 129/253] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:38.994021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.001335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.003038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.006570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.007091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.008079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:39.008591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:39.010194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:39.027183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 125/254] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:39.033236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.089204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:39.164327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aba0ac-23a9-4f62-83d7-339566de875d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:39.165517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810e962-41fe-4436-b954-70dc8cdfc9cd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:39.206064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.206724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.207009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.207307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.208147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] RBAC: Adding query filter params (role.id=cb8bcd18ccd341c1b9653ba56d25ceea, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=1627ceb579da41cc8b73d251ccd21999) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:39.208465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:39.209913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f18af5e-472e-42cb-811b-6c35a72f443b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:39.215612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.216632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.217053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.217438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.218774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:39.220420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:39.222803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 130/256] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:38 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:39.222941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 126/256] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:39 2022] GET /identity/v3/role_assignments?user.id=1627ceb579da41cc8b73d251ccd21999&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=cb8bcd18ccd341c1b9653ba56d25ceea => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:39.223716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.224614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22b1e1-98ff-4656-91bc-8be30f71a456 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.224908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22b1e1-98ff-4656-91bc-8be30f71a456 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.225190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b22b1e1-98ff-4656-91bc-8be30f71a456 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.228113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 127/257] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:39.232157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22b1e1-98ff-4656-91bc-8be30f71a456 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.298424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:39.365944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ba4c1-fa29-4977-939b-8ee137c95e63 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:39.412065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04694b84-09d4-4f76-a002-2f4914aca6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.412978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04694b84-09d4-4f76-a002-2f4914aca6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.413270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04694b84-09d4-4f76-a002-2f4914aca6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.413555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04694b84-09d4-4f76-a002-2f4914aca6c4 None admin] PATH_INFO: `/v3/users/cd63cffde1104f34be0683986641d472` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.415953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04694b84-09d4-4f76-a002-2f4914aca6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cd63cffde1104f34be0683986641d472)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:39.417670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04694b84-09d4-4f76-a002-2f4914aca6c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:39.419633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 131/258] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:39 2022] GET /identity/v3/users/cd63cffde1104f34be0683986641d472 => generated 283 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:39.421363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.422041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.423378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.446938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:39.448201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:39.453476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:39.466441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b22b1e1-98ff-4656-91bc-8be30f71a456 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:39.530054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 132/259] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:39.531956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.555024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.556300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.557619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.558877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/3ac67b1e20a84324a3ae8a037d1287ac/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.562941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5, user_id=3ac67b1e20a84324a3ae8a037d1287ac, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:39.568165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:39.591967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 128/260] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:39 2022] PUT /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/3ac67b1e20a84324a3ae8a037d1287ac/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:39.593175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.643240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:39.729013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24c4f7d-5cdf-4f67-8ad7-fb95c54d9c46 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:39.782777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183a94f-3ae8-44a8-970b-63ef8aa78e48 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:39.812046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.813157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.814597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.815085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.864006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:39.864829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:39.865244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:39.865522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:39.889424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:39.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:39.904611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:39.905406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:39.906966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 129/261] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:39.912703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:39.950544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:39.958603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:39.960120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:40.037895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:40.111946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 133/262] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:40.117653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:40.206081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71536541-4379-45b0-bc30-40b20acdd51f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:40.292068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:40.293116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.293603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.294100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.295122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:40.295652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:40.297245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:40.314411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 130/263] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:40.319950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:40.324222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a9a28-cb9a-47aa-a1b5-dc29e35c9be5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:40.389588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:40.390516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.390803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.391085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.396152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:40.397627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:40.399761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 134/264] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:40 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:40.412813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:40.470083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:40.474324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:40.541455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e08f242-60a0-40e5-9dc6-124ee2d1e42f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:40.624949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:40.625767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.626058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.626341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] PATH_INFO: `/v3/users/cd63cffde1104f34be0683986641d472` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.627602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] RBAC: Authorizing `identity:get_user(user_id=cd63cffde1104f34be0683986641d472)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:40.629056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:40.635410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 131/265] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:40 2022] GET /identity/v3/users/cd63cffde1104f34be0683986641d472 => generated 283 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:40.636540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:40.670683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279eda62-adb7-4c71-ba87-48c152548b76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:40.713795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:40.714806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.715581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.716108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.717644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=cd63cffde1104f34be0683986641d472) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:40.718152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:40.719796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a63f85f-d7fb-4fb7-b46f-0eab9347a7c5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:40.726533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 135/266] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:40 2022] GET /identity/v3/role_assignments?user.id=cd63cffde1104f34be0683986641d472&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:40.728052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe401a-f31c-4be3-92de-5c8db15ea95e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.728571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe401a-f31c-4be3-92de-5c8db15ea95e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.729040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe401a-f31c-4be3-92de-5c8db15ea95e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.730481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 136/267] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:40.732317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbfb43-349e-4d86-87da-5a6f4dbe8444 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.732819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbfb43-349e-4d86-87da-5a6f4dbe8444 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.733290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbfb43-349e-4d86-87da-5a6f4dbe8444 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.735060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 137/268] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:40.736660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1dbe8-4a8a-4e80-9249-7cf0405305f4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.737172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1dbe8-4a8a-4e80-9249-7cf0405305f4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.737646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf1dbe8-4a8a-4e80-9249-7cf0405305f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.755854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-adf1dbe8-4a8a-4e80-9249-7cf0405305f4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:40.757520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-adf1dbe8-4a8a-4e80-9249-7cf0405305f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:40.807393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0477f65-e620-41d4-9461-f38c9cbfd44b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:40.849063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:40.849754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.850044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.850323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.868971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:40.927333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 138/269] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:40.929004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:40.929490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:40.929960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:40.930200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:40.931727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:40.949247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:40.951565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:40.964659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.039932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 139/270] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:41.040945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.055080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 132/271] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:41.062103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.126443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.155774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a6d3-bb7d-4474-98d7-dce77b9a9845 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.196037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862104f-1e85-41c2-b087-08ac702a90a2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.198394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.199489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.199672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.199910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.213229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:41.222572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:41.223454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:41.224239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 140/272] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:41.224881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.236182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.251229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.252135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.252418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.252691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.256287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.258157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.260878 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 133/273] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:41.262219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.354008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b241f59-2046-4a82-83f4-59eb9e076da3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.381785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d4b422-4e8b-47e2-9e90-cfdd2120415c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.393993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.395185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.395716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.396198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.397671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.398528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.401658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.405067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 141/274] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:41.406282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.422365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.423054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.423378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.423657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.424657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:41.424966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.426953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.438266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 134/275] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:41.439052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.494184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.524039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca626857-eebd-4a38-8a5b-792623c639fa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.561408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.562546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.563107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.563617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.564717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] RBAC: Adding query filter params (role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=cd63cffde1104f34be0683986641d472) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:41.565330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.567075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.572626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f048d875-04d3-44b7-be94-0269b654d82d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.574249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 142/276] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/role_assignments?user.id=cd63cffde1104f34be0683986641d472&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=df14633e5004446c8c52b442c7d11a91 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:41.576252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.618013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.619119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.619667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.620142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.621176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.623497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.651303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 135/277] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:48:41 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:41.701933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e1dd4-0ef4-4775-b910-8a216f2ea517 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.746909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179b5039-40ce-4973-8bbf-8c3ffa609cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.748437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b5039-40ce-4973-8bbf-8c3ffa609cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.748703 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b5039-40ce-4973-8bbf-8c3ffa609cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.748981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b5039-40ce-4973-8bbf-8c3ffa609cd9 None admin] PATH_INFO: `/v3/users/688d82816e1b455bb99d0eb7a172fec0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.762950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.771200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b5039-40ce-4973-8bbf-8c3ffa609cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=688d82816e1b455bb99d0eb7a172fec0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.772836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179b5039-40ce-4973-8bbf-8c3ffa609cd9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.774919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 143/278] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/users/688d82816e1b455bb99d0eb7a172fec0 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:41.784610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:41.835764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:41.915038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cfb40d-03a9-478e-b268-68602aa139b4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:41.952191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:41.952861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:41.953146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:41.953427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:41.954264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, user.id=688d82816e1b455bb99d0eb7a172fec0, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:41.954581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:41.956119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad186f1d-9e19-4caf-9c8e-d96adfaf8056 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:41.961915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 136/279] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Jun 20 05:48:41 2022] GET /identity/v3/role_assignments?user.id=688d82816e1b455bb99d0eb7a172fec0&scope.system=all&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:42.017424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536469d-a7aa-49e1-903d-9f49bc77bf54 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.017676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536469d-a7aa-49e1-903d-9f49bc77bf54 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.018026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536469d-a7aa-49e1-903d-9f49bc77bf54 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.019553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 144/280] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:42.025543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8352e1-1d5e-45d1-96ea-6faa5aab9eed None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.026124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8352e1-1d5e-45d1-96ea-6faa5aab9eed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.026580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8352e1-1d5e-45d1-96ea-6faa5aab9eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.046734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7f8352e1-1d5e-45d1-96ea-6faa5aab9eed None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:42.048288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7f8352e1-1d5e-45d1-96ea-6faa5aab9eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:42.138536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 137/281] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:42.161939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536469d-a7aa-49e1-903d-9f49bc77bf54 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.248256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a2bd6-025a-4063-b764-f876e9da0652 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.248549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a2bd6-025a-4063-b764-f876e9da0652 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.248834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a2bd6-025a-4063-b764-f876e9da0652 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.250491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 138/282] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:42.268761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.269332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.269332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.289202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536469d-a7aa-49e1-903d-9f49bc77bf54 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:42.324288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:42.325564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:42.337542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:42.339483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.339778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.340061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.364180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:42.376669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:42.377583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:42.379063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 145/283] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:42.384260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:42.386889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.472200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 139/284] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 20 05:48:42.480381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:42.497966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.560763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb94e2b-cd15-47f0-9bf9-59fa2258fdde None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:42.597590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:42.598657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.599165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.599726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.600669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:42.601197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:42.602798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:42.612634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 146/285] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:42.618412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.724634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8ce696-edc9-4844-85d5-8aa25d6c7ee0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:42.734941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f84fc6-3195-4de7-b053-bd0534c1952b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:42.743821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:42.762926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:42.764159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.764674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.765165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] PATH_INFO: `/v3/users/1627ceb579da41cc8b73d251ccd21999` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.766584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] RBAC: Authorizing `identity:get_user(user_id=1627ceb579da41cc8b73d251ccd21999)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:42.768290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:42.774888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 147/286] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:42 2022] GET /identity/v3/users/1627ceb579da41cc8b73d251ccd21999 => generated 297 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:42.776381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.836694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:42.837524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.837820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.838102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.857341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:42.879343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:42.880251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:42.881637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 140/287] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:42.888696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.890113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0dae51-4e26-45a4-8382-645ab1c77cb2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:42.926271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:42.927534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:42.928022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:42.928514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:42.929896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:42.931672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:42.935612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:42.938995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 148/288] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:42 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:42.940171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:42.998134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:43.044795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242296-0092-486f-bf80-0e8636cbb2a6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:43.080116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:43.081071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:43.081361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:43.081641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/1627ceb579da41cc8b73d251ccd21999/roles/cb8bcd18ccd341c1b9653ba56d25ceea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:43.083723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] RBAC: Authorizing `identity:create_grant(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5, user_id=1627ceb579da41cc8b73d251ccd21999, role_id=cb8bcd18ccd341c1b9653ba56d25ceea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:43.085110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:43.104803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 149/289] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:48:42 2022] PUT /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/users/1627ceb579da41cc8b73d251ccd21999/roles/cb8bcd18ccd341c1b9653ba56d25ceea => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:43.131362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ae0ca-48a7-46d5-bff7-c733c2493773 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:43.189404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:43.190102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:43.190387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:43.190661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:43.191455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:43.191763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:43.193174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:43.209061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 141/290] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:43.214680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:43.361990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869efd12-06ed-470e-9cb8-1c0b0c725fde None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:43.376635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:43.398781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:43.400124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:43.400420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:43.400705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] PATH_INFO: `/v3/users/3ac67b1e20a84324a3ae8a037d1287ac` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:43.402068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] RBAC: Authorizing `identity:get_user(user_id=3ac67b1e20a84324a3ae8a037d1287ac)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:43.403654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:43.405764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 150/291] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:43 2022] GET /identity/v3/users/3ac67b1e20a84324a3ae8a037d1287ac => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:43.411426 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:43.504752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:43.537688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e01955b-bdfc-4302-b2c6-1b1d54ad63c8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:43.583838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:43.584751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:43.585182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:43.585601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:43.664622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:43.666298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:43.760377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:43.790492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 142/292] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:43.796043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:43.912285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaf104a-8227-42a9-96ba-29e6a9bf6160 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:43.946935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3842c7-f385-46c3-972d-8a0bfb75924c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:43.947843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3842c7-f385-46c3-972d-8a0bfb75924c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:43.948265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3842c7-f385-46c3-972d-8a0bfb75924c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:43.948545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3842c7-f385-46c3-972d-8a0bfb75924c None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:43.949746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3842c7-f385-46c3-972d-8a0bfb75924c None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:43.951269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3842c7-f385-46c3-972d-8a0bfb75924c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:43.953361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 151/293] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:43 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:43.960359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.126058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74aa6a76-0907-47c3-a28f-82d35d7a7473 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:44.179333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:44.180008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.180285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.180562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.181397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=3ac67b1e20a84324a3ae8a037d1287ac) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:44.181707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:44.183126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcce232-b1c2-45cb-8e0d-faf49dc10673 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:44.196717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 143/294] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:43 2022] GET /identity/v3/role_assignments?user.id=3ac67b1e20a84324a3ae8a037d1287ac&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:44.352933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f947b-eec8-4812-9b16-ce9852262abe None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.353279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f947b-eec8-4812-9b16-ce9852262abe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.353453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f947b-eec8-4812-9b16-ce9852262abe None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.355114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 152/295] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:44.360414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.360707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.360985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.379904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ab21c-77bd-426b-9766-7771fdb811e1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.380422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ab21c-77bd-426b-9766-7771fdb811e1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.380498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:44.380568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ab21c-77bd-426b-9766-7771fdb811e1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.381527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:44.382112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 153/296] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:44.389499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.389796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.390081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.411704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:44.412919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:44.431600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:44.471299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 144/297] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:44.478127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:44.495125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.502439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 154/298] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:44.531544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.605918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39686aaa-ede4-4e40-8414-b63411cbb094 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:44.643027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:44.644107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.644541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.644965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.645170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c59103-5326-4b2f-b8b3-e2d3e0044c16 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:44.656646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:44.665240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:44.666348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:44.667350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:44.669708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 145/299] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:44.675715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.683121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dbcb33-ff1d-4ed0-a2bd-017a02c108bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:44.683970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbcb33-ff1d-4ed0-a2bd-017a02c108bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.684253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbcb33-ff1d-4ed0-a2bd-017a02c108bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.684527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dbcb33-ff1d-4ed0-a2bd-017a02c108bb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.699147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dbcb33-ff1d-4ed0-a2bd-017a02c108bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-28dbcb33-ff1d-4ed0-a2bd-017a02c108bb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:44.710241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:44.711220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:44.712127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 155/300] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:44.713426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcc4ce-4af2-4ba9-bffe-e22a0d5fc703 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.713724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcc4ce-4af2-4ba9-bffe-e22a0d5fc703 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.714005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcc4ce-4af2-4ba9-bffe-e22a0d5fc703 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.716500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:44.720266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 156/301] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:44.721521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcc4ce-4af2-4ba9-bffe-e22a0d5fc703 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.765727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:44.794921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceee735-9157-479f-842f-453dc58372c5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:44.830429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcc4ce-4af2-4ba9-bffe-e22a0d5fc703 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:44.833451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:44.834130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.834420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.834702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.835478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:44.835793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:44.837163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cf8696-a388-4b91-ac6c-34613b69e73b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:44.850245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 146/302] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:44.850245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.850245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.850245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.865336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:44.866487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:44.868785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:44.869821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:44.869821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:44.870002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:44.870729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:44.871038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:44.872526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:44.881754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 157/303] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:44.882456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.951272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 147/304] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:44.951272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:44.979933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:44.986997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb9cf3-ebd8-41b8-9b7d-d52e22f0433a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:44.988901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:45.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.025920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.026205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.026483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] PATH_INFO: `/v3/users/735eaef4deee4e7a8976747bf0a371b0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.040822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] RBAC: Authorizing `identity:get_user(user_id=735eaef4deee4e7a8976747bf0a371b0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:45.042362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:45.049053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 158/305] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:44 2022] GET /identity/v3/users/735eaef4deee4e7a8976747bf0a371b0 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:45.049053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:45.063721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56109f9-fbbe-498f-bc55-74b867e17b51 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:45.100394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.101386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.101694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.101963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] PATH_INFO: `/v3/users/688d82816e1b455bb99d0eb7a172fec0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.103289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] RBAC: Authorizing `identity:get_user(user_id=688d82816e1b455bb99d0eb7a172fec0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:45.105005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:45.112150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 148/306] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:44 2022] GET /identity/v3/users/688d82816e1b455bb99d0eb7a172fec0 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:45.112976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:45.144662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883010e6-2aab-4fef-92e2-c43cbd84c685 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:45.182398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.183197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.183523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.183801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.185519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:45.186980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:45.193245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 159/307] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:45.193245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:45.210169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:45.221751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514443b8-fa5a-44a1-a631-1f62fd76e27b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:45.222959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:45.256941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.257600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.257886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.258167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.258997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] RBAC: Adding query filter params (role.id=cb8bcd18ccd341c1b9653ba56d25ceea, user.id=735eaef4deee4e7a8976747bf0a371b0, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:45.259335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:45.260768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:45.265735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 149/308] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Jun 20 05:48:45 2022] GET /identity/v3/role_assignments?user.id=735eaef4deee4e7a8976747bf0a371b0&scope.system=all&role.id=cb8bcd18ccd341c1b9653ba56d25ceea => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:45.266491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:45.316452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d87691c-ed68-4e92-ab5d-2e4eb7af4667 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:45.353791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d8735-5ef2-48ee-8b50-f1c9b1515ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.354672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8735-5ef2-48ee-8b50-f1c9b1515ed8 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.354963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8735-5ef2-48ee-8b50-f1c9b1515ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.355232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d8735-5ef2-48ee-8b50-f1c9b1515ed8 None admin] PATH_INFO: `/v3/system/users/688d82816e1b455bb99d0eb7a172fec0/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.357437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d8735-5ef2-48ee-8b50-f1c9b1515ed8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=688d82816e1b455bb99d0eb7a172fec0, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:45.358846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d8735-5ef2-48ee-8b50-f1c9b1515ed8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:45.370986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 160/309] 10.222.0.45 () {58 vars in 1334 bytes} [Mon Jun 20 05:48:45 2022] PUT /identity/v3/system/users/688d82816e1b455bb99d0eb7a172fec0/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:45.392233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8169fc-68dd-4d18-927d-9fed3b821890 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:45.434208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7011b9-1730-45ca-a3d8-ddf9ae56c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.435657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7011b9-1730-45ca-a3d8-ddf9ae56c4b8 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.435951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7011b9-1730-45ca-a3d8-ddf9ae56c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.436277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7011b9-1730-45ca-a3d8-ddf9ae56c4b8 None admin] PATH_INFO: `/v3/groups` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.437055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7011b9-1730-45ca-a3d8-ddf9ae56c4b8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:45.438580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7011b9-1730-45ca-a3d8-ddf9ae56c4b8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:45.458990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 150/310] 10.222.0.45 () {60 vars in 1134 bytes} [Mon Jun 20 05:48:45 2022] POST /identity/v3/groups => generated 229 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:45.640392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf194ab-3f85-401d-ab7c-69e8cf18df61 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.640903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf194ab-3f85-401d-ab7c-69e8cf18df61 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.641137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf194ab-3f85-401d-ab7c-69e8cf18df61 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.642657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 161/311] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:45.648130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.648671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.649147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.666272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:45.667807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:45.680392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:45.742131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 151/312] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:45.765382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf194ab-3f85-401d-ab7c-69e8cf18df61 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:45.864820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:45.877402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf194ab-3f85-401d-ab7c-69e8cf18df61 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:45.912640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:45.913568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:45.913857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:45.914136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:45.924630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:45.932550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:45.933431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:45.935181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 162/313] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:45.940356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:46.047681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9a7d6-bf4e-485c-a2e6-741da1e2f6a6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:46.083324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:46.084404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:46.084860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:46.085305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:46.086278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:46.086755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:46.088338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:46.097545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 152/314] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:46.103215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:46.208845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029eeadb-9d11-4529-8178-72ab7ff8c0e6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:46.243445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:46.244330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:46.244618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:46.244897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] PATH_INFO: `/v3/users/1627ceb579da41cc8b73d251ccd21999` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:46.246099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] RBAC: Authorizing `identity:get_user(user_id=1627ceb579da41cc8b73d251ccd21999)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:46.248168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:46.250162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 163/315] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:46 2022] GET /identity/v3/users/1627ceb579da41cc8b73d251ccd21999 => generated 297 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:46.255946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:46.268127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:46.367022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a89ea-fc54-40f3-8d05-df4f931e28fb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:46.401319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:46.401984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:46.402276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:46.402561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:46.476194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:46.477686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:46.488243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:46.583844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 153/316] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:46.592367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:46.616410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:46.697621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ded75-f9c6-4b89-af0d-2f536ceda236 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:46.732473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd7cb3-095a-40c8-86f7-236b768c10a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:46.733953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7cb3-095a-40c8-86f7-236b768c10a4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:46.734451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7cb3-095a-40c8-86f7-236b768c10a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:46.734921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd7cb3-095a-40c8-86f7-236b768c10a4 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:46.736347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7cb3-095a-40c8-86f7-236b768c10a4 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:46.738104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd7cb3-095a-40c8-86f7-236b768c10a4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:46.740308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 164/317] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:46 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:46.746125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:46.852810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3e7166-1d6b-41b0-b89c-aa242d8ea0ab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:46.871592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:46.889812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:46.890654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:46.891135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:46.891456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:46.892304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] RBAC: Adding query filter params (role.id=cb8bcd18ccd341c1b9653ba56d25ceea, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5, user.id=1627ceb579da41cc8b73d251ccd21999) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:46.892623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:46.894132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e2ec27-e835-4489-aa95-489008b7f92a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:46.900963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 154/318] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:48:46 2022] GET /identity/v3/role_assignments?user.id=1627ceb579da41cc8b73d251ccd21999&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=cb8bcd18ccd341c1b9653ba56d25ceea => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:47.343970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a87fa-07e8-43a8-bd2a-33e5504c3be7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.344340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a87fa-07e8-43a8-bd2a-33e5504c3be7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.344541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a87fa-07e8-43a8-bd2a-33e5504c3be7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.346306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 165/319] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:47.351200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1ae0d-6fb0-4b72-be7d-bd6fc218720a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.351523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1ae0d-6fb0-4b72-be7d-bd6fc218720a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.351805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb1ae0d-6fb0-4b72-be7d-bd6fc218720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.368877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4cb1ae0d-6fb0-4b72-be7d-bd6fc218720a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:47.370046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4cb1ae0d-6fb0-4b72-be7d-bd6fc218720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:47.438695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 155/320] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:47.460814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a87fa-07e8-43a8-bd2a-33e5504c3be7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:47.475932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfa3d4-288e-489e-845b-7ef1754c13b9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.476235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfa3d4-288e-489e-845b-7ef1754c13b9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.476752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfa3d4-288e-489e-845b-7ef1754c13b9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.477998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 156/321] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:48:47.482766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.483056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.483366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.500204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:47.501392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:47.562072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:47.580129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a87fa-07e8-43a8-bd2a-33e5504c3be7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:47.581757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 157/322] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:47.607027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:47.617468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:47.618393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.618691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.618974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.629336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:47.637009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:47.637958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:47.638739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 166/323] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:47.643913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:47.644815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:47.719351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a5345-3efc-432c-94ed-1511619a90c7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:47.753501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb7a18-e927-408f-a9ed-7660a6216011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:47.754309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7a18-e927-408f-a9ed-7660a6216011 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.754597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7a18-e927-408f-a9ed-7660a6216011 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.754876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb7a18-e927-408f-a9ed-7660a6216011 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.756643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf688f-66ab-490c-8f79-d58b2adedf7b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:47.765251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb7a18-e927-408f-a9ed-7660a6216011 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-afeb7a18-e927-408f-a9ed-7660a6216011 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:47.773040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:47.774026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:47.774819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 158/324] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:47.775928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2625c-f823-474b-a5a9-8d2a25237b64 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.775928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2625c-f823-474b-a5a9-8d2a25237b64 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.776044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2625c-f823-474b-a5a9-8d2a25237b64 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.777239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 159/325] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:47.779803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2625c-f823-474b-a5a9-8d2a25237b64 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:47.792736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:47.793405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.793695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.794502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.795244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:47.795908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:47.796972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbbdb79-2333-4b01-803c-e447f28f3c12 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:47.805103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 167/326] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:47.806281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.806576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.806858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.823494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:47.824134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:47.827582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:47.876887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:47.887184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2625c-f823-474b-a5a9-8d2a25237b64 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:47.890789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 168/327] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:47.891644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:47.928189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:47.928866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:47.929158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:47.929441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:47.930190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:47.930502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:47.931895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:47.940386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 160/328] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:47.944524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:47.989291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47773c9-9586-45ba-b469-cd31eca81a0b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:48.023540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:48.024401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:48.024690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:48.024990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] PATH_INFO: `/v3/users/735eaef4deee4e7a8976747bf0a371b0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:48.026160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] RBAC: Authorizing `identity:get_user(user_id=735eaef4deee4e7a8976747bf0a371b0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:48.027614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:48.029246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 169/329] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:47 2022] GET /identity/v3/users/735eaef4deee4e7a8976747bf0a371b0 => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:48.029989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:48.040082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:48.043394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400216c4-d939-42b3-995f-44f578709ac4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:48.077128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:48.077976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:48.078274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:48.078553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:48.079780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:48.081301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:48.082935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 161/330] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:48:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:48.083706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:48.107422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:48.128384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ae3f39-6691-4355-8ce2-9cf5d27a25f4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:48.162371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:48.163227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:48.163536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:48.163807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] PATH_INFO: `/v3/users/688d82816e1b455bb99d0eb7a172fec0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:48.164983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] RBAC: Authorizing `identity:get_user(user_id=688d82816e1b455bb99d0eb7a172fec0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:48.166416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:48.168067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 170/331] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:48 2022] GET /identity/v3/users/688d82816e1b455bb99d0eb7a172fec0 => generated 293 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:48.168812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:48.181253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307000f1-0d3f-40d4-b0ff-7a9603a83059 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:48.214299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:48.215157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:48.215466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:48.216272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] PATH_INFO: `/v3/system/users/735eaef4deee4e7a8976747bf0a371b0/roles/cb8bcd18ccd341c1b9653ba56d25ceea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:48.217743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=735eaef4deee4e7a8976747bf0a371b0, role_id=cb8bcd18ccd341c1b9653ba56d25ceea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:48.219096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:48.227665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 162/332] 10.222.0.45 () {58 vars in 1334 bytes} [Mon Jun 20 05:48:48 2022] PUT /identity/v3/system/users/735eaef4deee4e7a8976747bf0a371b0/roles/cb8bcd18ccd341c1b9653ba56d25ceea => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:48.228647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:48.245982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:48.275411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1348759d-fbc6-48de-8034-ddc7accfebf1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:48.315137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d1f0c1-2a39-4304-b9a8-4f44eeecdfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:48.315862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1f0c1-2a39-4304-b9a8-4f44eeecdfd9 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:48.316156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1f0c1-2a39-4304-b9a8-4f44eeecdfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:48.316433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d1f0c1-2a39-4304-b9a8-4f44eeecdfd9 None admin] PATH_INFO: `/v3/groups` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:48.317168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1f0c1-2a39-4304-b9a8-4f44eeecdfd9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:48.318648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d1f0c1-2a39-4304-b9a8-4f44eeecdfd9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:48.336565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 171/333] 10.222.0.45 () {60 vars in 1134 bytes} [Mon Jun 20 05:48:48 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:48:48.351482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:48.372709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59fbaba-7850-4815-bcfc-9d3265919184 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:48.413041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:48.413636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:48.413926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:48.414208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:48.415067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] RBAC: Adding query filter params (role.id=10f492ea4ccb4aa0b627ead79bc02dea, user.id=688d82816e1b455bb99d0eb7a172fec0, scope.system=all) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:48.415443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:48.416913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c6bb3e-a835-487d-84bb-fb0b5fabedab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:48.431182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 163/334] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Jun 20 05:48:48 2022] GET /identity/v3/role_assignments?user.id=688d82816e1b455bb99d0eb7a172fec0&scope.system=all&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 510 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:50.440323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102185a-c39a-44df-9f02-c6bbad7f5556 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:50.440840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102185a-c39a-44df-9f02-c6bbad7f5556 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:50.441179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102185a-c39a-44df-9f02-c6bbad7f5556 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:50.442926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 172/335] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:50.457410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db7a1-d520-42eb-af01-bb1681ec2a1e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:50.457900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db7a1-d520-42eb-af01-bb1681ec2a1e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:50.457971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3db7a1-d520-42eb-af01-bb1681ec2a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:50.475725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5b3db7a1-d520-42eb-af01-bb1681ec2a1e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:50.476935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5b3db7a1-d520-42eb-af01-bb1681ec2a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:50.560939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 164/336] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:50.596094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102185a-c39a-44df-9f02-c6bbad7f5556 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:50.651890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b76d2-f167-4e36-8add-cd45a40a14a2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:50.652211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b76d2-f167-4e36-8add-cd45a40a14a2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:50.652462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b76d2-f167-4e36-8add-cd45a40a14a2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:50.653891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 165/337] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:50.658855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:50.659143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:50.659451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:50.683632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:50.685359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:50.773529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 166/338] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:50.775959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102185a-c39a-44df-9f02-c6bbad7f5556 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:50.800109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:50.803190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:50.840887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:50.841716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:50.842006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:50.842285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:50.855622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:50.872291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:50.880098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:50.880966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:50.882170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 173/339] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:50.887307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:50.948879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191e884b-0564-4212-991e-70ccea73f1cb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:50.991152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e879d7-afc6-413e-abf3-423aec09a995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:50.992038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e879d7-afc6-413e-abf3-423aec09a995 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:50.992328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e879d7-afc6-413e-abf3-423aec09a995 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:50.992609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e879d7-afc6-413e-abf3-423aec09a995 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.005203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e879d7-afc6-413e-abf3-423aec09a995 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-40e879d7-afc6-413e-abf3-423aec09a995 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:51.013694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:51.015893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 167/340] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:48:51.017859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24980b8-6985-4c0d-8aa9-dfc9e3da30e6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.020627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24980b8-6985-4c0d-8aa9-dfc9e3da30e6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.020627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24980b8-6985-4c0d-8aa9-dfc9e3da30e6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.020627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 168/341] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:51.024327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.027241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.027241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.047146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182d4a5-7268-45f1-b608-2e70f84c9d94 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.060149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:51.061590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:51.117548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:51.118509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.118942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.119553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.121589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:51.122819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:51.124448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:51.143531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 174/342] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:51.144597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.180114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:51.195276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 169/343] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:51.205616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.229835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:51.278979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc21b6a-a839-4f1d-a95d-49342a327101 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.321118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:51.322039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.322475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.322901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.323852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:51.325144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:51.326684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:51.348843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 175/344] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:51.349964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.423243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3a8e61-0ef2-4b1b-9b15-ff20917a2bcf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.482854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:51.483772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.484219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.484663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] PATH_INFO: `/v3/users/735eaef4deee4e7a8976747bf0a371b0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.486074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] RBAC: Authorizing `identity:get_user(user_id=735eaef4deee4e7a8976747bf0a371b0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:51.487748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:51.489926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 170/345] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/users/735eaef4deee4e7a8976747bf0a371b0 => generated 293 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:51.490844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.497571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597995dc-a7f7-4aea-8de3-85e71b924f99 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.518810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:51.565183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:51.565929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.566215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.566490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.570947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:51.600187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:51.608894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:51.609804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:51.610706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:51.615720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 176/346] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:51.616638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.616749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ccb552-be9d-4018-9343-86dcf6ff11ae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.654186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed440b-22f4-4c49-aab8-1244716a0615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:51.654897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed440b-22f4-4c49-aab8-1244716a0615 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.655191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed440b-22f4-4c49-aab8-1244716a0615 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.655493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed440b-22f4-4c49-aab8-1244716a0615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.730191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed440b-22f4-4c49-aab8-1244716a0615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:51.732433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed440b-22f4-4c49-aab8-1244716a0615 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:51.744278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1244ce9f-3ab2-424d-bded-dcd9eb779476 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.779975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:51.780935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.781417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.781838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.782856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] RBAC: Adding query filter params (role.id=cb8bcd18ccd341c1b9653ba56d25ceea, user.id=735eaef4deee4e7a8976747bf0a371b0, scope.system=all) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:51.783343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:51.785140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:51.796561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:51.799845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 177/347] 10.222.0.45 () {56 vars in 1308 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/role_assignments?user.id=735eaef4deee4e7a8976747bf0a371b0&scope.system=all&role.id=cb8bcd18ccd341c1b9653ba56d25ceea => generated 510 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:51.800857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.841683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 171/348] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:51.843331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504b13-affd-4221-b12e-b89c6e6b5930 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:51.843766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504b13-affd-4221-b12e-b89c6e6b5930 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:51.844188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86504b13-affd-4221-b12e-b89c6e6b5930 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:51.845925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 172/349] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:51.847228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504b13-affd-4221-b12e-b89c6e6b5930 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:51.869856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:51.960155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86504b13-affd-4221-b12e-b89c6e6b5930 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.977823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f21f4-1032-41ad-b551-93b7e7892a1b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:51.998910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb644ef5-5e0e-4fc2-a969-d421a0077458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.000116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb644ef5-5e0e-4fc2-a969-d421a0077458 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.000546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb644ef5-5e0e-4fc2-a969-d421a0077458 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.001012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb644ef5-5e0e-4fc2-a969-d421a0077458 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.002355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb644ef5-5e0e-4fc2-a969-d421a0077458 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.004098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb644ef5-5e0e-4fc2-a969-d421a0077458 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.006768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 173/350] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:52.008343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.008782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.009203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.026768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:52.029058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:52.034528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.035209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.035524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.035808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.036575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:52.036888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.038287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.055226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 178/351] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:52.058143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.081164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.133542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 174/352] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:52.134662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.244566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d658ceaa-4c70-45f2-9339-29a00c5bed68 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:52.275634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee6cfe9-5ffd-4332-9de0-3aaf894f98ee None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:52.279854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.292751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.293421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.294290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.294521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.318117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.319371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.319861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.320392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.324171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.328461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.330189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.332211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 179/353] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:52.333184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.369172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.370733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.445553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d4759-18ab-4a3f-a30e-b5688659b42f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:52.489564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c9a5c4-b177-469a-a57e-82ae8c7bbfd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.490374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9a5c4-b177-469a-a57e-82ae8c7bbfd1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.490661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9a5c4-b177-469a-a57e-82ae8c7bbfd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.491123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c9a5c4-b177-469a-a57e-82ae8c7bbfd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.495092 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 175/354] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:52.496059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.504383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c9a5c4-b177-469a-a57e-82ae8c7bbfd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-44c9a5c4-b177-469a-a57e-82ae8c7bbfd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:52.515069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:52.516163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:52.516968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 180/355] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:52.518063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec6bf0-8c31-4880-9481-a2b39a2cb122 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.518324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec6bf0-8c31-4880-9481-a2b39a2cb122 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.518612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec6bf0-8c31-4880-9481-a2b39a2cb122 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.522483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 181/356] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:52.522483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec6bf0-8c31-4880-9481-a2b39a2cb122 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.554000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.574549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.630786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6945e75-efeb-47a5-88e7-6b0d0188e110 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:52.664983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec6bf0-8c31-4880-9481-a2b39a2cb122 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:52.672480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.673146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.673435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.673715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.674707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:52.675024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.676503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.683954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 176/357] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:52.684839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.707811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dcdbb3-1ed4-4651-8980-57e38eb7e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.708620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdbb3-1ed4-4651-8980-57e38eb7e158 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.708909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdbb3-1ed4-4651-8980-57e38eb7e158 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.709191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dcdbb3-1ed4-4651-8980-57e38eb7e158 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.710544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcdbb3-1ed4-4651-8980-57e38eb7e158 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.712116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dcdbb3-1ed4-4651-8980-57e38eb7e158 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.714466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 182/358] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:52.716322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.716614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.716894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.741232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:52.742535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:52.817224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da7b4-0d38-4e0a-b0a0-0c1156f8cb05 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:52.828397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 183/359] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:52.828397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.856696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.864410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:52.886191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:52.886879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.887185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.887565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.888436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:52.888751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:52.890243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de41e52b-931e-4304-aeb9-5f8f288ee5e8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:52.899199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 177/360] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:52.900680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93270f0-ec62-4d09-aeaf-abe34453374e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:52.900972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93270f0-ec62-4d09-aeaf-abe34453374e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:52.901249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93270f0-ec62-4d09-aeaf-abe34453374e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:52.902578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 178/361] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:52.907725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93270f0-ec62-4d09-aeaf-abe34453374e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:52.959522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d589ca85-ce7c-42a2-b776-76d44e6e6433 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.002154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.003585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.004027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.004450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.005928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:53.008391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:53.010214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 184/362] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:53.010827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.050124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93270f0-ec62-4d09-aeaf-abe34453374e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.089770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799c2e1-5a0d-442b-ba7f-e0dd3fa74bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.090652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799c2e1-5a0d-442b-ba7f-e0dd3fa74bbb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.090947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799c2e1-5a0d-442b-ba7f-e0dd3fa74bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.091227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799c2e1-5a0d-442b-ba7f-e0dd3fa74bbb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.103990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799c2e1-5a0d-442b-ba7f-e0dd3fa74bbb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4799c2e1-5a0d-442b-ba7f-e0dd3fa74bbb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:53.113237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:53.114134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:53.114925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 179/363] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:53.116098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.120783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.120783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.123313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:53.138063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c179a-da52-447e-bbbd-dfb12b4703d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.139873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:53.141289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:53.184309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.185030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.185378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.185669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.192058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:53.234138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 180/364] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:53.236157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.281218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:53.283175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:53.361980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07cacf-a05e-43b1-b73a-5b723987cbad None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.412240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:53.416001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.416676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.416966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.417248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.418102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:53.418421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:53.421243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:53.426923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 181/365] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=df14633e5004446c8c52b442c7d11a91 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:53.429111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 185/366] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:53.429990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.431587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.491677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:53.566844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f79deb-f440-4fa8-a44f-3346ee4696d9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.572636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f944b2-608d-426e-aa68-f97198d3bafd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.614016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.615412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.615412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.615412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.621641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.622331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.622625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.623080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.624207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:53.624523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:53.626044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:53.627736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:53.635965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 182/367] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:53.636907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:53.638040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:53.639035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:53.639968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.640682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 186/368] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:53.643101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.719975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:53.783120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98391503-12a3-41b3-a165-e54a6540884d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.785063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:53.788666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce72de-7395-450b-83bc-2366ea75c2f3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.825254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.831275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.831275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.831275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.836002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:53.836886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:53.837172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:53.837451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:53.838693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:53.840183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:53.849021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 183/369] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:53.849021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:53.919475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:53.921007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:53.985839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aac5b7-ec81-4d19-bbd1-8bb97b275b99 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:53.990952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.039593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.040555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.042278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.042734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.044647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:54.045040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.046796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.053478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 187/370] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:54.054518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.061695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 184/371] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:54.062691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.103195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.163148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa5f0ee-b60e-4e41-9997-f735f403e7c0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.180766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca729afd-d32f-4a7b-b4ff-a3c56d867987 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.200446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.201632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.202115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.202587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.204067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.205760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.207628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 188/372] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:54.208587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.234780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.235786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.235996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.236247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.237605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.238767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.240202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.242435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 185/373] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:54.243185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.340990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef707ef-c6bf-4111-8516-41c1ede03a4a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.365233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.369087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08abe7f-bbef-40d8-bb29-af72f74ef331 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.377511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.378460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.378956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.379462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.408314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.408986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.409276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.409557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.410502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:54.410825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.412298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.419680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 186/374] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:54.420487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.456113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.457983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.478658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.570713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 189/375] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:48:54.576903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.617139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7a108-63c8-4a6b-aa3d-66af231ba54f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.713401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.714295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.714589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.714871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.716198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.717607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.723625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 187/376] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:54.725623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.730070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbcfc5-cab8-401e-bb58-9da1962dee85 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.751181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.772419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:54.775840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.778004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.778577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.779063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] PATH_INFO: `/v3/projects/b72f99dc88224a27868e009fcc68d646` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.780538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] RBAC: Authorizing `identity:get_project(project_id=b72f99dc88224a27868e009fcc68d646)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.782204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.784955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 190/377] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/projects/b72f99dc88224a27868e009fcc68d646 => generated 344 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:54.792723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:54.856568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e4482-c3f4-4d3a-84df-6e9aa882d256 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.898081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.898668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.898956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.899237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.900103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:54.900420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.901850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.914033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 188/378] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=df14633e5004446c8c52b442c7d11a91 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:54.923785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58d5e4-9d83-412e-a8db-ad693e2ab5bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:54.966462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b876991f-a69a-44cb-b75d-c84b73bb31f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:54.967359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876991f-a69a-44cb-b75d-c84b73bb31f2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:54.967654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876991f-a69a-44cb-b75d-c84b73bb31f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:54.967937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b876991f-a69a-44cb-b75d-c84b73bb31f2 None admin] PATH_INFO: `/v3/groups/f7a4f038261f417c904f70adae9d23c1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:54.974747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876991f-a69a-44cb-b75d-c84b73bb31f2 None admin] RBAC: Authorizing `identity:get_group(group_id=f7a4f038261f417c904f70adae9d23c1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:54.976273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b876991f-a69a-44cb-b75d-c84b73bb31f2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:54.979439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 191/379] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/groups/f7a4f038261f417c904f70adae9d23c1 => generated 229 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:54.984651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:55.081479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881af516-64fd-4928-b501-1b47bd57a056 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.081824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881af516-64fd-4928-b501-1b47bd57a056 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.082113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881af516-64fd-4928-b501-1b47bd57a056 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.083718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 192/380] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:55.090830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.091459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.091967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.097022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:55.104472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d6693a-a7ec-49e9-ba87-49bace80b46d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:55.111571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:55.113204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:55.144111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:55.144788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.145079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.145361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.146276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] RBAC: Adding query filter params (group.id=f7a4f038261f417c904f70adae9d23c1, role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=b72f99dc88224a27868e009fcc68d646) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:55.146599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:55.148235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:55.159344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 189/381] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:48:54 2022] GET /identity/v3/role_assignments?group.id=f7a4f038261f417c904f70adae9d23c1&scope.project.id=b72f99dc88224a27868e009fcc68d646&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:55.173138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:55.209144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 193/382] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:48:55.248820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:55.376947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aec3b2-fde5-4b1b-9c6d-5cccf2daeb6f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:55.414296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:55.415029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.415347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.415632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.426847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:55.434901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:55.435813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:55.436602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 190/383] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:55.443777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:55.587496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882878a-c390-4117-9787-1a3460db76e4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:55.625816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:55.626856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.627375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.627884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.628841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:55.629354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:55.630960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4060af7-dbdc-4e6e-88d0-2cdc9c85389d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:55.641354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 194/384] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:55.647158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:55.673314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:55.768800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29350157-bf62-4aad-b8c0-5b79d82b2513 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:55.811810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb32291-d004-4dee-87d4-44640c8f1f7b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.812199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb32291-d004-4dee-87d4-44640c8f1f7b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.812486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb32291-d004-4dee-87d4-44640c8f1f7b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.812615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:55.813433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.813692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.813971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.814179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 195/385] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:55.815809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:55.817279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:55.819198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 191/386] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:55 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:55.828855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:55.829360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:55.829665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:55.829929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:55.869514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:55.870731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:55.937411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:55.941716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:55.957421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c68f1-bf7f-4d3c-814f-0d052168b0f0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:55.998686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:55.999899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:56.000374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:56.000860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:56.002250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:56.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:56.006199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 192/387] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:55 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:56.010379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 196/388] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:56.016152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:56.093793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:56.141321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea6c31-c991-4fc4-9024-e1760b3d102b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:56.187212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:56.188644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:56.189140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:56.189823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:56.192261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c253b10dbca4a15b11c1d888597ca3b, group_id=427dad8a3ef1477fb83d7aa09e9ef61f, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:56.194047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:56.214364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 193/389] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Jun 20 05:48:56 2022] PUT /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:56.292346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9df28-0671-45f8-8797-c2f55ffb0762 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:56.340096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:56.341520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:56.341811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:56.342087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:56.360368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:56.368873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:56.369879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:48:56.370753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 197/390] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:56.377404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:56.409167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:56.498903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6063114d-8807-4324-ad9a-97eff1bb0fd8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:56.540415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:56.541457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:56.541949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:56.542455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:56.543483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:56.544014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:56.545584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:56.558801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 194/391] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:56.565220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:56.627169 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:56.738787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a37a15-a295-4897-ae15-54f4e0b5779d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:56.781440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:56.782229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:56.782518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:56.782818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:56.784028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:56.785488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:56.787448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 198/392] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:56 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:56.793141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:56.911366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:56.937291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e1e82c-12d1-4f1b-ae34-016eb3652a2a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:56.977645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e25b90-5797-4191-b604-c1dc5ba680e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:56.978523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25b90-5797-4191-b604-c1dc5ba680e5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:56.978807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25b90-5797-4191-b604-c1dc5ba680e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:56.979081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25b90-5797-4191-b604-c1dc5ba680e5 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:56.980286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e25b90-5797-4191-b604-c1dc5ba680e5 None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:56.981815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e25b90-5797-4191-b604-c1dc5ba680e5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:56.983861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 195/393] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:56 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:56.989680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:57.120079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:57.123740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8aba7c-dd00-43ec-9898-1cbb701fb8f1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:57.168573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd44009-1408-45d9-87f1-8130f706ccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:57.169819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd44009-1408-45d9-87f1-8130f706ccc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:57.170326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd44009-1408-45d9-87f1-8130f706ccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:57.170801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd44009-1408-45d9-87f1-8130f706ccc9 None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:57.177039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd44009-1408-45d9-87f1-8130f706ccc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c253b10dbca4a15b11c1d888597ca3b, group_id=427dad8a3ef1477fb83d7aa09e9ef61f, role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:57.178727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd44009-1408-45d9-87f1-8130f706ccc9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:57.211262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 199/394] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Jun 20 05:48:56 2022] PUT /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/df14633e5004446c8c52b442c7d11a91 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:57.415076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d87468-0d82-4abb-9d1a-7d0e486c468a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:57.415429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d87468-0d82-4abb-9d1a-7d0e486c468a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:57.415638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d87468-0d82-4abb-9d1a-7d0e486c468a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:57.417214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 196/395] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:57.422338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:57.422632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:57.422989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:57.440648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:57.441832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:57.515891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 200/396] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:57.550424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d87468-0d82-4abb-9d1a-7d0e486c468a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:57.665956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d87468-0d82-4abb-9d1a-7d0e486c468a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:57.702835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:57.703687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:57.703967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:57.704239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:57.724160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:57.731888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:57.732784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:57.734038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 197/397] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:57.739217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:57.847144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:57.858313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9112509-fa9e-4f73-9541-ec7d993786ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:57.906788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:57.907871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:57.908405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:57.908888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:57.909839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:57.910361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:57.911980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8473389-51e1-4799-9b4a-58a7c1b5b26c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:57.921866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 201/398] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:57.926983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:57.945327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:58.075009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2bfa9-21c9-475a-8939-d6bf332d341b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.087368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a85a-af13-4dbc-bca0-31d2d74f2ff1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.088053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a85a-af13-4dbc-bca0-31d2d74f2ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.088564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a2a85a-af13-4dbc-bca0-31d2d74f2ff1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.094865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 202/399] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:58.097026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620112-806b-4592-be63-23cc4c96da6a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.097318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620112-806b-4592-be63-23cc4c96da6a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.097599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d620112-806b-4592-be63-23cc4c96da6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.114003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.114863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.115151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.115465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.116746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:58.118365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:58.120484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 198/400] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:57 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:58.128128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.135356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0d620112-806b-4592-be63-23cc4c96da6a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:58.136842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d620112-806b-4592-be63-23cc4c96da6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:58.212661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:58.217644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 203/401] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:58.219750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a395-140a-47af-b71a-44902a280ab4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.220180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a395-140a-47af-b71a-44902a280ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.221096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a71a395-140a-47af-b71a-44902a280ab4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.223060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 204/402] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:58.231269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.231269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.231269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.253225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8791aeb9-f60f-49f7-acbe-747b8ebdaebb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.257971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:58.259165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:58.291666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.292905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.293909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.294185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.295873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:58.297588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:58.299706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 199/403] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:58.300744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.351348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 205/404] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:58.352516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.377729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:58.423700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c528c40-49cb-441b-8314-fe2b8e2d964c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.469640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.471439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.472288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.472779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.483049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d583ecd-da59-41d7-823a-a043ac4394b0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.487128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:58.497306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:58.498182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:48:58.498976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:58.501974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 200/405] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:58.507371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.508959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:58.534734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.535732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.536015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.536341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.539204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] RBAC: Authorizing `identity:create_grant(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5, group_id=427dad8a3ef1477fb83d7aa09e9ef61f, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:58.541406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:58.567341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 206/406] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Jun 20 05:48:58 2022] PUT /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:58.570513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.709700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a77335-940c-468e-baba-b98cba44cd62 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.751363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:58.774183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773d7c73-4c8a-47aa-a0b6-205cfdffb489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.775492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d7c73-4c8a-47aa-a0b6-205cfdffb489 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.775982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d7c73-4c8a-47aa-a0b6-205cfdffb489 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.776479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773d7c73-4c8a-47aa-a0b6-205cfdffb489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.789984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd31e68-bf4a-417b-b0e4-6fa307ed9f6d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.790750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773d7c73-4c8a-47aa-a0b6-205cfdffb489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-773d7c73-4c8a-47aa-a0b6-205cfdffb489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:58.798535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:58.799488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:58.800271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 201/407] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:58.801446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8e95-663a-4b34-bee5-4971c89a32a9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.801673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8e95-663a-4b34-bee5-4971c89a32a9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.801954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad8e95-663a-4b34-bee5-4971c89a32a9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.803418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 202/408] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:58.807508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8e95-663a-4b34-bee5-4971c89a32a9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.851165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.853106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.853655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.854154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.855158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:58.855725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:58.857822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c5393-fa91-421f-bfb3-2d8e82e66868 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:58.869401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 207/409] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:58.875981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.876272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.876549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.887362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:58.900141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:58.901605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:58.932117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad8e95-663a-4b34-bee5-4971c89a32a9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:58.970966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:58.971980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:58.972447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:58.972909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:58.974204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:58.974770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:58.976457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:58.986697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 203/410] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:48:58.987601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:58.999338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 208/411] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:59.000898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.090015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:59.110858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90d428-10ee-4a86-ada0-7b2f7e06ffdd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.137509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6608c-ecb1-4f44-8228-643da262b788 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.139338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:59.152400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.153766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.154311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.154795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.156332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.158074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.164661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 204/412] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:59.165831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.180317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.181576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.181867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.182148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] PATH_INFO: `/v3/groups/f7a4f038261f417c904f70adae9d23c1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.183652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] RBAC: Authorizing `identity:get_group(group_id=f7a4f038261f417c904f70adae9d23c1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.185427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.188114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 209/413] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:48:58 2022] GET /identity/v3/groups/f7a4f038261f417c904f70adae9d23c1 => generated 229 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:59.189129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.298247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe79db5-97cc-49f4-a88e-081d03c45545 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.299996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1f363-a983-4c02-a523-7e8f2cfe30af None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.332475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:59.339429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.341380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.341960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.342464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.343594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.344289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.344396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.344658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.344932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.346892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.353988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 210/414] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:59 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:59.355084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.356402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:48:59.364873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:48:59.365748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:48:59.366545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:48:59.369187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 205/415] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:48:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:48:59.370222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.439980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:59.480633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60e27a5-1f34-4286-9e91-c37ffe52f2eb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.495130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb92fbf-3dc5-4fce-9d99-ca903803d82d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.532464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee23041-f62b-4e5a-a1f1-a36f69bb4257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.533382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23041-f62b-4e5a-a1f1-a36f69bb4257 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.533486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.533667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23041-f62b-4e5a-a1f1-a36f69bb4257 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.533947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23041-f62b-4e5a-a1f1-a36f69bb4257 None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.534327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.534611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.534883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] PATH_INFO: `/v3/projects/b72f99dc88224a27868e009fcc68d646` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.536096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] RBAC: Authorizing `identity:get_project(project_id=b72f99dc88224a27868e009fcc68d646)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.536385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee23041-f62b-4e5a-a1f1-a36f69bb4257 None admin] RBAC: Authorizing `identity:create_grant(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5, group_id=427dad8a3ef1477fb83d7aa09e9ef61f, role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.537525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.537809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee23041-f62b-4e5a-a1f1-a36f69bb4257 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.539392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 211/416] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:48:59 2022] GET /identity/v3/projects/b72f99dc88224a27868e009fcc68d646 => generated 344 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:59.540600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.559364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 206/417] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Jun 20 05:48:59 2022] PUT /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5/groups/427dad8a3ef1477fb83d7aa09e9ef61f/roles/df14633e5004446c8c52b442c7d11a91 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:59.560931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509867-42df-4221-b5ac-8b6e2f02907a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.561223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509867-42df-4221-b5ac-8b6e2f02907a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.561531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49509867-42df-4221-b5ac-8b6e2f02907a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.567001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 207/418] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:48:59.567001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509867-42df-4221-b5ac-8b6e2f02907a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.640843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:59.692794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf275506-9252-4828-9649-0c918b381d3a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.708522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49509867-42df-4221-b5ac-8b6e2f02907a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.726591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:48:59.738787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.739974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.740472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.740914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.741872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] RBAC: Adding query filter params (name=member) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:48:59.742257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.744260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79cbef-15db-4d6d-a69b-7ce9fc2fd23a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.753491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 212/419] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:48:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:48:59.754749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.755042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.755705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.756595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.756877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.757152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] PATH_INFO: `/v3/projects/b72f99dc88224a27868e009fcc68d646/groups/f7a4f038261f417c904f70adae9d23c1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.759185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] RBAC: Authorizing `identity:create_grant(project_id=b72f99dc88224a27868e009fcc68d646, group_id=f7a4f038261f417c904f70adae9d23c1, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:48:59.759811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:48:59.760646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:48:59.779331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:48:59.780457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 208/420] 10.222.0.45 () {58 vars in 1442 bytes} [Mon Jun 20 05:48:59 2022] PUT /identity/v3/projects/b72f99dc88224a27868e009fcc68d646/groups/f7a4f038261f417c904f70adae9d23c1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:48:59.780656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:48:59.781356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.869056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 213/421] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:48:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:48:59.904409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:48:59.916927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b7643d-6472-4c04-9c6e-4a6d5766d10e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:48:59.960274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:48:59.961214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:48:59.961505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:48:59.962259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.015559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:00.024667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:00.051763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:00.053232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:00.081715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a8d2e-9dcd-48e3-a81c-5f1456010fbb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:00.121740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:00.122910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.123438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.123910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.135226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:00.146633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:00.147541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:49:00.148341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:00.151053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 214/422] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:48:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:00.157205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:00.184456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 209/423] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:48:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:00.190803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:00.279572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:00.333478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f16817-bafc-4982-9440-baed8c0e3d66 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:00.375927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa28b74-de7f-4172-a74f-68681643739c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:00.398360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:00.399270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.399554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.399833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.401910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:00.404001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:00.405850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 210/424] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:49:00 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:00.412097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:00.412767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:00.441859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:00.442536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.442955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.443240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.444023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:00.444335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:00.445715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:00.464494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 215/425] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:00.469180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:00.621270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c08dff-f34c-416d-b4a3-89627caf973b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:00.626501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45806ed-3699-4568-bf7b-03e8f339e8b4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:00.690554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:00.692553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.692839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.693114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.696119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:00.697564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:00.699199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 211/426] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:49:00 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:00.704971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:00.716454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dcf4dc-c948-4157-9194-8be9bf79f62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:00.717112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcf4dc-c948-4157-9194-8be9bf79f62b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.717399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcf4dc-c948-4157-9194-8be9bf79f62b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.717677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dcf4dc-c948-4157-9194-8be9bf79f62b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.729917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:00.778584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:00.804497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcf4dc-c948-4157-9194-8be9bf79f62b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:00.806725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dcf4dc-c948-4157-9194-8be9bf79f62b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:00.884248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4378215-7b17-4259-9913-0b86bd8ca31f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:00.914721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 216/427] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:00.916365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6a345-ef8b-4565-a16f-454a56052f7e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.916907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6a345-ef8b-4565-a16f-454a56052f7e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.917398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6a345-ef8b-4565-a16f-454a56052f7e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.919215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 217/428] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:00.927944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6a345-ef8b-4565-a16f-454a56052f7e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:00.931671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:00.932782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.932782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.932975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.934307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:00.934626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:00.936071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c475eb3-e5c2-43df-90e1-8db0b2b7ad19 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:00.942427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 212/429] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:49:00 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:00.951842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:00.952196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:00.952424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:00.992469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:00.993689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:01.027720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:01.041238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6a345-ef8b-4565-a16f-454a56052f7e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:01.079154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:01.080477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.081008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.081500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] PATH_INFO: `/v3/projects/6c253b10dbca4a15b11c1d888597ca3b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.082960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] RBAC: Authorizing `identity:get_project(project_id=6c253b10dbca4a15b11c1d888597ca3b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:01.084890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:01.093080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 218/430] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:49:00 2022] GET /identity/v3/projects/6c253b10dbca4a15b11c1d888597ca3b => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:01.100337 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:01.139079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 213/431] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:01.200990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:01.203164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:01.234447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34244ea9-05a5-460c-a8b2-1e36e948856b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:01.271735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:01.272982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.273470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.273947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.277186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:01.279448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:01.286122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 219/432] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:01.287747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:01.308769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:01.318558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf826b98-b0ed-4d35-8d20-33065357f67c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:01.360999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:01.361819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.362086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.362363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.378651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:01.389520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:01.390434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 20 05:49:01.391222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:01.392933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 214/433] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:01.397429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:01.409950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c80edf-8cb4-4502-88a5-2bc04bd4b958 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:01.461225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:01.462450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.463205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.463729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.464749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=6c253b10dbca4a15b11c1d888597ca3b) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:01.465305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:01.467723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:01.479374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:01.479752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 220/434] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=6c253b10dbca4a15b11c1d888597ca3b&role.id=df14633e5004446c8c52b442c7d11a91 => generated 611 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:01.527621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7b8d0c-2c38-4672-9a0d-856583801bcb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:01.567551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:01.568512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.569036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.569456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.570369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] RBAC: Adding query filter params (name=member) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:01.570827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:01.572408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411ef48d-3d38-4238-9bcb-53cf5bb9f717 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:01.587489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 215/435] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:01.589559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:01.695074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:01.716512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486da5f-ca92-421e-a79e-fdaea34820bf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:01.753783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:01.754437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.754720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.754999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.830728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365adec9-5aac-4ebc-9012-0b31922bec89 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.831026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365adec9-5aac-4ebc-9012-0b31922bec89 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.831371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365adec9-5aac-4ebc-9012-0b31922bec89 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.832745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 216/436] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:01.837655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:01.837947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:01.838228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:01.839485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:01.841685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:01.857303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:01.858495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:01.945706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 217/437] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:49:01.955367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:01.971005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:01.974972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 221/438] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:01.980215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.039175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.093779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7639c-c8e5-4169-a0ed-4b2877030d53 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.097107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2064ec8-7c44-405e-abc1-620dea0e2f2b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.132529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.133453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.133739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.134019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.134135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.134847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.135139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.135354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.135475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.137328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.143931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 222/439] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:02.144945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.146453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:02.155293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:02.156178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:49:02.156985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:02.159544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 218/440] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:02.162261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.198518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.250432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4342464-ca49-42b9-811a-385989a387cb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.260790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fabefe-77f5-4bcb-8e1a-49dbacd9154e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.274538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.285008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc387e2-bb35-4073-aee1-312e48fdc930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.285859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc387e2-bb35-4073-aee1-312e48fdc930 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.286144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc387e2-bb35-4073-aee1-312e48fdc930 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.286411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc387e2-bb35-4073-aee1-312e48fdc930 None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.287616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc387e2-bb35-4073-aee1-312e48fdc930 None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.289067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc387e2-bb35-4073-aee1-312e48fdc930 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.290675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 223/441] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:02.291852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab4af8-fc83-4d0b-8887-eada8888bd9e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.292137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab4af8-fc83-4d0b-8887-eada8888bd9e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.292412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab4af8-fc83-4d0b-8887-eada8888bd9e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.293637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 224/442] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:02.296182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab4af8-fc83-4d0b-8887-eada8888bd9e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.301136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.301807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.302096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.302378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.303134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:02.303477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.304860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b4cac4-20e1-4b82-aa5b-9cadac6cb7b0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.313262 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 219/443] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:02.314497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.314792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.315072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.331404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:02.332587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:02.395303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab4af8-fc83-4d0b-8887-eada8888bd9e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.397313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 220/444] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:02.398114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.419607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.438712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.439736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.440023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.440298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.441441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=10f492ea4ccb4aa0b627ead79bc02dea, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:02.441757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.443187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.459281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 225/445] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=10f492ea4ccb4aa0b627ead79bc02dea => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:02.459281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.513879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d65b-7d73-4057-8c21-0d5343349925 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.522060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.551572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.552347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.552631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.552914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.561544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850cbc69-964d-4ecb-bd1c-35e267c99676 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.600958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.602121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.602407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.602993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.617998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:02.626172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:02.627059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:02.628306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 226/446] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:02.631974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.633288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.633926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.687983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.754602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2561515-4ff3-428b-bc78-645ca6d3f75f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.755962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 221/447] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:02.761473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.769534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.796197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.796872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.797164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.797444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.798193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:02.798504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.799939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.808642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 227/448] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:02.814461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.870854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62476a9-eb0b-4bb0-bfae-47c081c74281 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.905671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.906531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.906815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.907093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] PATH_INFO: `/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.908318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] RBAC: Authorizing `identity:get_project(project_id=9f423cb92f7244abbc0d9ebd69b4e6f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:02.909763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:02.911609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 222/449] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/projects/9f423cb92f7244abbc0d9ebd69b4e6f5 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:02.916569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c5ad00-9de3-48d0-b8e6-9e4f5265e213 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:02.917015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:02.919475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:02.950087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:02.950724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:02.951015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:02.951314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:02.989743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:03.014961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b04def3-eb45-49c0-89a2-cbe13cb7742a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:03.017335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:03.018891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:03.047896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:03.048731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:03.049011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:03.049285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] PATH_INFO: `/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:03.050467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] RBAC: Authorizing `identity:get_group(group_id=427dad8a3ef1477fb83d7aa09e9ef61f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:03.051916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:03.053510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 223/450] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/groups/427dad8a3ef1477fb83d7aa09e9ef61f => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:03.059086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:03.114829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 228/451] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 20 05:49:03.120180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:03.122773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:03.156677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ae156-b2a7-4fa9-96b8-ca36c305dbfb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:03.190568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:03.191322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:03.191671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:03.191954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:03.192777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] RBAC: Adding query filter params (group.id=427dad8a3ef1477fb83d7aa09e9ef61f, role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:03.193087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:03.194497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:03.200797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 224/452] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:49:03 2022] GET /identity/v3/role_assignments?group.id=427dad8a3ef1477fb83d7aa09e9ef61f&scope.project.id=9f423cb92f7244abbc0d9ebd69b4e6f5&role.id=df14633e5004446c8c52b442c7d11a91 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:03.217453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5ed5a-6d15-451b-a69b-ce421efed365 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:03.254121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:03.255009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:03.255324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:03.255606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] PATH_INFO: `/v3/projects/b72f99dc88224a27868e009fcc68d646` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:03.256877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] RBAC: Authorizing `identity:get_project(project_id=b72f99dc88224a27868e009fcc68d646)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:03.258368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:03.260138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 229/453] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:49:03 2022] GET /identity/v3/projects/b72f99dc88224a27868e009fcc68d646 => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:03.266498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:03.275540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:03.371980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aff6cb-422d-45f4-943f-4418cbe8c3f8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:03.404853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22397bf6-37ba-4749-b5be-fc1d70d6d189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:03.405716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397bf6-37ba-4749-b5be-fc1d70d6d189 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:03.405998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397bf6-37ba-4749-b5be-fc1d70d6d189 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:03.406275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22397bf6-37ba-4749-b5be-fc1d70d6d189 None admin] PATH_INFO: `/v3/groups/f7a4f038261f417c904f70adae9d23c1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:03.407486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397bf6-37ba-4749-b5be-fc1d70d6d189 None admin] RBAC: Authorizing `identity:get_group(group_id=f7a4f038261f417c904f70adae9d23c1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:03.408912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22397bf6-37ba-4749-b5be-fc1d70d6d189 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:03.410524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 225/454] 10.222.0.45 () {56 vars in 1180 bytes} [Mon Jun 20 05:49:03 2022] GET /identity/v3/groups/f7a4f038261f417c904f70adae9d23c1 => generated 229 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:03.416421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:03.489733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:03.520260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6641b7ed-30bb-4921-8083-ccc9ac0882f4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:03.554520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:03.555170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:03.555504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:03.555786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:03.556616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] RBAC: Adding query filter params (group.id=f7a4f038261f417c904f70adae9d23c1, role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=b72f99dc88224a27868e009fcc68d646) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:03.556932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:03.558379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc56bb4-1de9-491b-9237-1a7804ec4c24 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:03.564630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 230/455] 10.222.0.45 () {56 vars in 1376 bytes} [Mon Jun 20 05:49:03 2022] GET /identity/v3/role_assignments?group.id=f7a4f038261f417c904f70adae9d23c1&scope.project.id=b72f99dc88224a27868e009fcc68d646&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:06.296407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897daca-e63d-4c9f-b0e8-8a329350e31c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.296927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897daca-e63d-4c9f-b0e8-8a329350e31c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.297000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897daca-e63d-4c9f-b0e8-8a329350e31c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.298676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 226/456] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:06.303486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ec84-2fdc-4fbf-bcbe-f3a3ea66530d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.303874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ec84-2fdc-4fbf-bcbe-f3a3ea66530d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.304057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ec84-2fdc-4fbf-bcbe-f3a3ea66530d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.308949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7196e-e5ca-4c47-b383-cee2b8532173 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.309240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7196e-e5ca-4c47-b383-cee2b8532173 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.309521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad7196e-e5ca-4c47-b383-cee2b8532173 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.312739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 227/457] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:06.320130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.320419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.320700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.331745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1294ec84-2fdc-4fbf-bcbe-f3a3ea66530d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:06.332959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1294ec84-2fdc-4fbf-bcbe-f3a3ea66530d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:06.352476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:06.353683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:06.462111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 231/458] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:06.465828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b661c-864e-47f9-b20f-2b9b392d9556 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.466123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b661c-864e-47f9-b20f-2b9b392d9556 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.466405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793b661c-864e-47f9-b20f-2b9b392d9556 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.469464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 232/459] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:06.472145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 228/460] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:06.473457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.473753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.474036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.493129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:06.496945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:06.498153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:06.505198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:06.574921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:06.584050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 233/461] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:06.585018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:06.629129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c2f1f6-ac70-4edf-9cc8-390b7c3cb54b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:06.665305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:06.666337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.666626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.666905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.678016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:06.679625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:06.698894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:06.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:06.700806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:06.700806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:06.700806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:06.700806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:06.705152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 229/462] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:06.705525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd4a31-4869-4706-afe6-e92fca739722 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:06.706002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:06.743378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86488079-3373-4ecf-a799-74907a16adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:06.744181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86488079-3373-4ecf-a799-74907a16adef None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.744569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86488079-3373-4ecf-a799-74907a16adef None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.744734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86488079-3373-4ecf-a799-74907a16adef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.755820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86488079-3373-4ecf-a799-74907a16adef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:06.757277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86488079-3373-4ecf-a799-74907a16adef None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:06.769478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-86488079-3373-4ecf-a799-74907a16adef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:06.780764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:06.781790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:06.782844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:06.782844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:06.782844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:06.782844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:06.782844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 234/463] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:06.783697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c75a-657d-4fb3-ba67-70d7c921e412 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.783697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c75a-657d-4fb3-ba67-70d7c921e412 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.783937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665c75a-657d-4fb3-ba67-70d7c921e412 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.785318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 235/464] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:06.788223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c75a-657d-4fb3-ba67-70d7c921e412 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:06.830615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1792f-5593-4a25-aed8-9a2976088a84 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:06.855023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:06.876123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:06.876935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.877222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.877503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.890834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:06.892351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ae8f10ed-7f4f-41bf-9246-9dea6cea2a28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:06.911341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:06.912594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:06.913413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:06.913413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:06.913413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:06.913413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:06.913413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 230/465] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:06.914252 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.914350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.914596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.917463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665c75a-657d-4fb3-ba67-70d7c921e412 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:06.931573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:06.932945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:06.958582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:06.959268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:06.959572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:06.959853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:06.960589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:06.960905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:06.962488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:06.977998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 236/466] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:06.979162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.019304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 231/467] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:07.019304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.054882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.103041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087c96b3-5af4-4e57-b698-744197414f20 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.117355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.124238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e102703-7760-4bee-a2bc-dbed1d054ef4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.147783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.148741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.149240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.149701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.150604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:07.151102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.153213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.160991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.161742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.162034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.162314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.163042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:07.163381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.164749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.167529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 237/468] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:07.169053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.177894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 232/469] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:07.178721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.279599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4b3085-f189-4b44-ab4d-774af35e79f7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.284269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.288313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548cee4f-ef09-483c-a0bb-b5d47abb0cfd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.314327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.314988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.315297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.315579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.316280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.317736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.321739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.322538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.322823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.323101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.334808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.336269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.343410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 238/470] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:49:07 2022] POST /identity/v3/users => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:07.343410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:07.363091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:07.364174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:07.365201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:07.365201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:07.365201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:07.365201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:07.366850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 233/471] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:07.367890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.377157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.465701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b4d99-e3b2-413d-a7d1-80a74f1e0ae5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.483505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69807c26-4acc-4e12-991e-659d2802fd4c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.504851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d8a96-f006-409a-b601-a73e7b988c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.505540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d8a96-f006-409a-b601-a73e7b988c72 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.505831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d8a96-f006-409a-b601-a73e7b988c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.506111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d8a96-f006-409a-b601-a73e7b988c72 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.506823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d8a96-f006-409a-b601-a73e7b988c72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.508793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d8a96-f006-409a-b601-a73e7b988c72 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.525670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad68f629-7308-4703-b8c9-835398451f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.526588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68f629-7308-4703-b8c9-835398451f0f None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.527060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68f629-7308-4703-b8c9-835398451f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.528301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad68f629-7308-4703-b8c9-835398451f0f None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.528301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68f629-7308-4703-b8c9-835398451f0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.530158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad68f629-7308-4703-b8c9-835398451f0f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.530560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 239/472] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:49:07 2022] POST /identity/v3/users => generated 248 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:07.532025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26135c21-8535-44a0-8097-e06c761beafa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.532323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26135c21-8535-44a0-8097-e06c761beafa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.532605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26135c21-8535-44a0-8097-e06c761beafa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.534177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 240/473] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:07.534944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26135c21-8535-44a0-8097-e06c761beafa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.564733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 234/474] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:49:07 2022] POST /identity/v3/users => generated 248 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:07.566065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.566377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.566656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.581943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.590549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:07.592394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:07.643066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.700205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 235/475] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:07.708171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26135c21-8535-44a0-8097-e06c761beafa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.728775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.773668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.774603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.775036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.775538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.776500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:07.777536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.779219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.793037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 241/476] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:07.801365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.868742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637750-52bf-487f-8279-12d2f016f280 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.913840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.914863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.915410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.915910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.916857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.918526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:07.922049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:07.928553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.sql.core [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 05:49:07.928553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 20 05:49:07.928553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [parameters: {'id': '4733f834b17f4d17933f68849c2f8dfd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 20 05:49:07.928553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80161) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 05:49:07.929198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cacb733-a1ad-4340-9f67-c0332069aa41 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() Jun 20 05:49:07.953442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return ref.to_dict() Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 05:49:07.954419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application yield resource Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application next(self.gen) Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise self.value Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application session.commit() Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 05:49:07.956110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.session.flush() Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application compat.raise_( Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 05:49:07.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 05:49:07.958556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application conn.query(q) Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application result.read() Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 05:49:07.961150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [parameters: {'id': '4733f834b17f4d17933f68849c2f8dfd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:07.963301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 20 05:49:07.964884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:07.965898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 236/477] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Jun 20 05:49:07 2022] POST /identity/v3/roles => generated 144 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:07.965898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:07.984194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:07.985631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:07.986117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:07.986665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:07.987702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:07.989459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:08.006382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:08.036646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 242/478] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:49:07 2022] POST /identity/v3/users => generated 249 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:08.075520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2337f4-43fb-41f9-ae85-86809e8de04d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:08.112606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14347fd-a8d0-4c87-89da-400781c77388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:08.113432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14347fd-a8d0-4c87-89da-400781c77388 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:08.113716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14347fd-a8d0-4c87-89da-400781c77388 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:08.113995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14347fd-a8d0-4c87-89da-400781c77388 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:08.124790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14347fd-a8d0-4c87-89da-400781c77388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c14347fd-a8d0-4c87-89da-400781c77388 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:08.133128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:08.134107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:08.134898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 237/479] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:08.140974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:08.291071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0e354-8b8e-404a-87bb-727eb31c635c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:08.328089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:08.329532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:08.330014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:08.330486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:08.331454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:08.332000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:08.333594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2fb24e-8cfa-42db-bede-26ddfb43b6a5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:08.343868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 243/480] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:09.860931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128b29-ea23-4488-a048-09d5aecb4ba3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:09.861463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128b29-ea23-4488-a048-09d5aecb4ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:09.861539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4128b29-ea23-4488-a048-09d5aecb4ba3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:09.863155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 238/481] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:09.868349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aac149-2db8-409d-8f50-8d434ea038be None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:09.869148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aac149-2db8-409d-8f50-8d434ea038be None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:09.869643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aac149-2db8-409d-8f50-8d434ea038be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:09.896193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-86aac149-2db8-409d-8f50-8d434ea038be None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:09.897681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-86aac149-2db8-409d-8f50-8d434ea038be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:09.966311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:09.989832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 244/482] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:10.036723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128b29-ea23-4488-a048-09d5aecb4ba3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.130200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:10.170069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880379f-bf54-4c6b-82ae-3db8f619d9be None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.170693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880379f-bf54-4c6b-82ae-3db8f619d9be None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.171177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9880379f-bf54-4c6b-82ae-3db8f619d9be None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.173028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 245/483] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:10.178425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f6d11-5e9e-4190-b0c9-44b96deb58cd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.178710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f6d11-5e9e-4190-b0c9-44b96deb58cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.178987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f6d11-5e9e-4190-b0c9-44b96deb58cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.184857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4128b29-ea23-4488-a048-09d5aecb4ba3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.205050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-809f6d11-5e9e-4190-b0c9-44b96deb58cd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:10.206521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-809f6d11-5e9e-4190-b0c9-44b96deb58cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:10.238490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.239771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.240283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.240762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.261974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:10.270691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:10.271578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:10.272385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:10.274994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 239/484] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:10.280909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.335377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 246/485] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:49:10.344818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fed5e-71f9-4c57-92b4-8980519d5b2c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.345719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fed5e-71f9-4c57-92b4-8980519d5b2c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.346201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791fed5e-71f9-4c57-92b4-8980519d5b2c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.347987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 247/486] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:10.354494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.355090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.355446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.383187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:10.384807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:10.430241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0753f9-2765-4b19-925d-a93585a42780 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.451716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:10.467585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.468676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.469440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.470434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.471443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:10.472011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:10.472824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 248/487] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:49:10.473683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.474211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:10.490244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 240/488] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:10.491793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.536000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:10.596710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b5b65-ce0f-4757-9a2b-59dfaa11193f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.624188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520931d-4fee-4db0-9fba-1753a3192a82 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.643122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.644320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.644879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.645474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.658228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:10.664381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.665181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.665467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.665744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:10.666077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:10.666923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 249/489] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:10.668164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.683761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:10.685241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5a26c1e5-ab23-44f3-8866-20735dfc66dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:10.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:10.704261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:10.705327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:10.705327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:10.705327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:10.705327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:10.710972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 241/490] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:10.710972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eb017-e98c-4722-8f56-a008172d8bca None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.710972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eb017-e98c-4722-8f56-a008172d8bca None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.710972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27eb017-e98c-4722-8f56-a008172d8bca None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.710972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 242/491] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:10.710972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eb017-e98c-4722-8f56-a008172d8bca None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.749913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:10.782101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5769-1f23-4a50-aa3a-1f9986c30ae8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.791564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:10.820571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.820571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.820571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.820571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.832505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:10.834631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27eb017-e98c-4722-8f56-a008172d8bca None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:10.840273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:10.841166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:10.841957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:10.846513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 250/492] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:10.847368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.873508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.874639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.875057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.875368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:10.876913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:10.878709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176aa8b8-1b3d-4d90-85fb-935da1d39793 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:10.893966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 243/493] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:10.893966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.893966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.893966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.910856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:10.912081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:10.955139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8761c94-594c-40b6-b8a1-b0fae2a4fead None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:10.985159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 244/494] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:10.986070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:10.996937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:10.997875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:10.998373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:10.998834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:10.999778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:11.000347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.002082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.015198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 251/495] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:11.016053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.022043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.044194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.095457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88265a3-d70b-4689-a17f-d6b67cc64e7d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.113935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec6d41c-5d72-4cbf-bb25-9f8461f8d1ab None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.127674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.128335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.128621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.128898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.129638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:11.129946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.131330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dbb7ee-4502-4596-bae1-e2469f42b3e7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.145051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 245/496] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:11.145051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c186f1-bd92-4e49-ac58-2358b2ea6ed2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.145051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c186f1-bd92-4e49-ac58-2358b2ea6ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.145051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c186f1-bd92-4e49-ac58-2358b2ea6ed2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.145051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 246/497] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:11.145051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c186f1-bd92-4e49-ac58-2358b2ea6ed2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.150077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.150869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.151156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.151514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.161563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.163029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:11.181261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:11.182242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:11.183872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:11.183872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:11.183872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:11.183872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:11.183872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 252/498] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:11.184183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.236898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.251083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c186f1-bd92-4e49-ac58-2358b2ea6ed2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.266173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.281298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027b82de-862d-439e-bb10-8d8f68812330 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.283604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.284398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.284684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.284963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.295479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:11.303041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:11.304129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 247/499] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:11.305023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.317599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.318390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.318675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.318953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.326477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.327953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-93202cdc-b82f-4a36-923b-fd16e73a39b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:11.343886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:11.344923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:11.346273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 253/500] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:11.346960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.347033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.347344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.363063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:11.364284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:11.407437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166541be-077c-4e46-af58-5ff279190f5f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.429507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 254/501] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:11.431463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.440773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.441567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.441854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.442130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.455762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.457649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.465652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:11.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:11.477173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:11.478010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:11.478010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:11.478010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:11.478010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:11.478010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 248/502] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:11.478644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.505605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.536027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c608f6-c9cd-4763-b385-7ac98ac4d611 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.568578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.569229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.569509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.569781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.570494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:11.570797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.572273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.577466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22257b5a-7cf0-484a-a51e-cc63effaa0c9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.583019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 255/503] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:11.583821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.610496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.611155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.611469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.611742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.612472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:11.612776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.614125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.622216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 249/504] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:11.622983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.683475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.687112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723d0406-d83e-4ea3-b5b4-f588da2a8b3b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.714851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.721281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.721425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f8c4d-7ec1-454d-895c-3a9daaae5cee None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.721949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.722238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.722517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.723386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:11.723701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.725149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.755422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.756213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.756499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.756776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.760588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 256/505] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:11.761393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.769577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.771031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:11.788182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:11.789028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:11.789852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:11.789852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:11.789852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:11.789852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:11.789852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 250/506] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:11.790115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.862334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c5b22c-e8f1-421f-b317-e598ab4014d9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.890577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b462b1e9-6aa3-4840-a2ae-74d30d47c3ba None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:11.895813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.896472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.896758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.897042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.897758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:11.898068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.899500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.904073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:11.911137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 257/507] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:11.912129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.924125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:11.924922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:11.925205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:11.925479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:11.932747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:11.934238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:11.950881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:11.951811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:11.952727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 251/508] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:11.953290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:11.961270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.012654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6841d2-7016-4011-bfd7-de264308f35a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.044600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.045485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.045775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.046048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.051164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aeb743-b835-4bcd-b73b-d896e6473543 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.056436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.057895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:12.075002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:12.075860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:12.076712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:12.076712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:12.076712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:12.076712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:12.076712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 258/509] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:12.076968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.085896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.086542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.086830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.087109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.128318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.153473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.154963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.170705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.173628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a8a10b-e692-409c-adb4-31e386dad2d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.209017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.209595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.209884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.210164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.210880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:12.211192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.212586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.224613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 259/510] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:12.225404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.254594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 252/511] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:12.255688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.322363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d3dad3-198a-481c-b9fe-cd8dd9d5dfcf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.331687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.356593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.356776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6730864-6195-4f2c-a4ac-bf1a7a906cc2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.357396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.357683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.357956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.364835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.366308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:12.381292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:12.382203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:12.383078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 260/512] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:12.383660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.394231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.394905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.395178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.395483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.396312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:12.396617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.398030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.404124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.433885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 253/513] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:12.434670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.482807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e6cb0-a918-42ea-9f9d-e07993c41657 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.518205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.518865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.519146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.519450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.520166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:12.520475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.521861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.530838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddeef1b-2429-4f2c-af8b-e2ecb0e2bef5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.537380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 261/514] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:12.537380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.559046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.564141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.564811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.565100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.565378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.566081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.567564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.586952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 254/515] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:49:12 2022] POST /identity/v3/users => generated 247 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:12.587775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.633050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.656521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b071a94-5720-4400-bfbc-74a75ac1a036 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.694737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.695581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.695867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.696147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.702100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726836e8-ba71-4a70-a7ab-0db0f59c8be0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.707690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.709180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:12.733526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:12.734555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:12.735649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:12.735649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:12.735649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:12.735649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:12.739303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 262/516] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:12.740175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.750704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.751417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.751707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.751994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.752864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:12.753176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.754610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.775385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 255/517] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:12.778565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.836225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.857526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ca8286-e151-4355-a674-76121e4ac19f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.891575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:12.892441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551f8b97-44e6-4d44-ae11-6ace4370951b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:12.898370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.899024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.899339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.899622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.928613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:12.929870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:12.930436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:12.930904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:12.939292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.940979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:12.958871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:12.959991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 05:49:12.961065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:12.967821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 256/518] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:12.968790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:12.970339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:12.971848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.068213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ccb9-e7c3-4014-94c3-7509a44f83ff None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.075480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.081960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 263/519] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:13.082802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.108313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.108979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.109266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.109546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.110267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:13.110581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.111988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.116351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.127887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 257/520] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:13.127887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.191038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc73eaa-dd21-426e-9f66-21992204b556 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.224494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8453b8b-bf6b-497c-9353-cc67c3a0f972 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.226837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.227517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.227804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.228084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.258880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.259596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.259882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.260164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.261061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:13.261374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.262796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.284495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 258/521] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:13.284495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.302255 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.304074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.309602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.337858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.388695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da031e4b-cf6a-4fae-b26b-fdf38c890d86 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.420358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 264/522] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:13.420358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.429862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.430666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.431004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.431319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.442245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.443718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:13.461411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:13.462395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:13.463300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:13.463300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:13.463300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:13.463300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:13.463300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 259/523] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:13.463557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.527996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7a2d47-3234-427b-a6a0-1aaa3def201d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.540643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.558611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.563805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.564800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.565381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.565965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.566960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ee7335-09ac-4adb-927a-265d5097e3fd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.573124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.575223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:13.590733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:13.591802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:13.593066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:13.596885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 265/524] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:13.597590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.608506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.609265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.609554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.609828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.687289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.687745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.699334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c809b-8a3f-4a51-b453-d9c0017a60c1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.736934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.737727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.738013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.738286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.750120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.751596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.768564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:13.769384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:13.770217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:13.771109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:13.771109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:13.771109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:13.771109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:13.771109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 266/525] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:13.772110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.799140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 260/526] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:13.800008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.823286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:13.882493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5d07f6-2e2a-4145-9d9c-0e3d9e013bc7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.909698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4dd5a2-0a8e-4824-99e0-d3158b8ad33c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:13.917937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.918598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.918881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.919159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.919906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:13.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.921585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.933844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 267/527] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:13.934637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:13.950499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:13.951328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:13.951616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:13.951914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:13.952701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:13.953214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:13.954415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:13.973142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 261/528] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:13.973966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.012211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.042129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ecf63-1b41-4e9e-a0b4-e60bc6b4b0d8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.049045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.081903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649d375c-2771-4091-9fbe-13638338d73d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.082474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.083133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.083463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.083782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.084517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.084820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.086219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.098990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 268/529] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:14.099832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.122269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.123062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.123375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.123654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.134338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.135747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:14.154744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:14.155652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:14.156532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:14.156532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:14.156532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:14.156532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:14.156532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 262/530] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:14.156797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.208355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286cad56-8ded-41cf-a3ce-ae994c9a95f0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.244952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.246711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.247009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.247336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.252673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.255127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.256711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.265731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb021b-abdc-4c1b-9317-2b6e50bcff8f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:14.271178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:14.272032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 269/531] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:14.278150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.301466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.306088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.306751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.307127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.307425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.308709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=ec34108c1ae24ef699265e0be32e99cf) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.309131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.310441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.316116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 263/532] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/role_assignments?user.id=ec34108c1ae24ef699265e0be32e99cf&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:14.323371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.402993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2df138-5baf-42d1-a202-99ecff8e7a8a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.449446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.450937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.451338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.451699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.461663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.463899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.465698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba327c-29bb-44e0-8586-f0049e3f73ff None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:14.482748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:14.483675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:14.484562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 270/533] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:14.490289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.509863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.510588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.510942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.511780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.512529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.512952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.514318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.533681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 264/534] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:14.534590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.551621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.583550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.602143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531dd59c-8861-48d8-ab8b-ac72832925a1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.638573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.639627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.640148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.640625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.641615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.642045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0b5f6-8e1d-4369-9ef1-7e13cd259d9e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.642340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.644036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.662663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 271/535] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:14.663885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.677763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.678463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.678730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.679007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.679828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.680145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.681572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0afbd99-e8d2-4c4c-8827-d7cf224176b4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.698519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 265/536] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:14.699715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b4cc6-f073-41b6-b760-00ea5b18d83a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.700016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b4cc6-f073-41b6-b760-00ea5b18d83a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.700296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b4cc6-f073-41b6-b760-00ea5b18d83a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.701571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 266/537] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:14.702260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b4cc6-f073-41b6-b760-00ea5b18d83a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.768873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe93b59-3771-4546-9733-92a7517ffc4b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.791844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.805280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.806085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.806376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.806652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.814070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.815582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:14.830220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:14.831096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:14.832376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:14.832791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b4cc6-f073-41b6-b760-00ea5b18d83a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.836217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 272/538] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:14.837068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:14.842904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:14.869023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.869929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.870269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.871480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.872581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=00892de83c5444c7a38b921615b2f5ed) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.872897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.874574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56066a9d-8e12-4a6d-b285-17e7bc81af26 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:14.881051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 267/539] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/role_assignments?user.id=00892de83c5444c7a38b921615b2f5ed&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:14.882571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.882946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.883372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.906305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:14.907715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:14.955613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdfa711-1aac-4dd1-b89e-a2fa704cc304 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:14.995541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:14.996362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:14.996717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:14.997084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:14.998182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=89c777401892485099d5448c506debb0) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:14.998571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:14.999263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 268/540] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:15.000766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.008821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:15.023584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 273/541] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/role_assignments?user.id=89c777401892485099d5448c506debb0&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:15.033514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.112022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:15.128805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:15.155768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76f7ab-5c8f-43cc-b90d-91115f3cae2a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:15.174432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4512eca9-c8fa-4476-a966-d2badac4f820 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:15.193392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:15.194072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:15.194361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:15.194644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:15.195716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:15.196039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:15.198087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:15.217740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 269/542] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:15.221031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.243709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:15.244869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:15.245895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:15.246372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:15.257522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:15.266978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:15.267909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:15.268698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:15.271419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 274/543] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:15.283299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.386413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a033d3-7c35-4b55-ac2d-eb556b93cf05 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:15.415008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:15.424921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:15.425598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:15.425884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:15.426163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:15.426919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:15.427231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:15.428777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:15.443054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 275/544] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:15.444614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.503068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff8bc3-2aeb-4c1f-8fd3-94f9116ea105 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:15.555394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:15.557039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf5974-17c8-4ed4-9843-ec3fc603ba0b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:15.571934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:15.572899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:15.573922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:15.574381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:15.575429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=fcefc528820f4d95b779c8972b46bd0f) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:15.575927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:15.577536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:15.587907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 270/545] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:15 2022] GET /identity/v3/role_assignments?user.id=fcefc528820f4d95b779c8972b46bd0f&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:15.606585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:15.607439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:15.607734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:15.608014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:15.623446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:15.624927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:15.650389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:15.651334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:15.652222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:15.652222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:15.652222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:15.652222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:15.658028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 276/546] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:15.659393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.789425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9154892-7e4c-4a1a-99e9-b9c947bdc445 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:15.831271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:15.831271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:15.831271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:15.831271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:15.831602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:15.831602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:15.831602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:15.840860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 271/547] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:15.846411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:15.898833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:15.975573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb426ab-dbb5-4430-85c2-f4ac16264ce2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:16.024145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:16.024952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:16.025241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:16.025526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:16.038107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:16.039665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:16.057381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:16.058910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:16.060302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 277/548] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:16.064810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:16.122954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:16.198020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d0704-6e7d-40c2-bec4-4324bba76aeb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:16.270335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:16.271023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:16.271356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:16.275334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:16.275334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:16.275334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:16.275334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ba27d6-0c83-47dd-8859-3d94c2b61a06 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:16.306972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 272/549] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Jun 20 05:49:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:16.312499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:16.509332 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e2bf2-c363-4309-a77c-55958677b3e0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:16.511211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:16.577185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11e0b5-f455-4de0-a876-d2bc37013faf None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:16.577487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11e0b5-f455-4de0-a876-d2bc37013faf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:16.577688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11e0b5-f455-4de0-a876-d2bc37013faf None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:16.579289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 273/550] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:16.585033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:16.585332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:16.585614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:16.594407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:16.604237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:16.604237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:16.604237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:16.611988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:16.613199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:16.776556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 274/551] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:16.779378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:16.780878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:16.804748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:16.888187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:16.918430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 278/552] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 606 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:16.933435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:16.935580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:16.973107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bce0b57-5000-4289-9109-74dcf4aaae8d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.022811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.023653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.023995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.024216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.041412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:17.057476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:17.058510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:17.059355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 275/553] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:17.066318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca59737-deb5-4ae0-97c3-704108ff265f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.072866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.140481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.142837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.143124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.143433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.168505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.169997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ca550225-4d35-49f5-83ad-0028f1b0935a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:17.202030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:17.203171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:17.204356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:17.204356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:17.204356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:17.204356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:17.206186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 279/554] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:17.211943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385cd2-6ca1-400b-a3b5-ae55f10f90de None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.212110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:17.212242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385cd2-6ca1-400b-a3b5-ae55f10f90de None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.212534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2385cd2-6ca1-400b-a3b5-ae55f10f90de None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.214643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 280/555] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:17.215528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385cd2-6ca1-400b-a3b5-ae55f10f90de None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.220327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b422a33a-a78c-4333-8015-f0d1aba09023 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.265096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.265776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.266064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.266588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.267946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:17.268277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.269928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7cf9ce-21fb-44b5-8a10-1fea7dd2ae80 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.277928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:17.282282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 276/556] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:17.283869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.284314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.284744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.304761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:17.306182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:17.334520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2385cd2-6ca1-400b-a3b5-ae55f10f90de None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.370701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.371386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.371675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.371954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.372673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:17.372984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.374366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.377696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 277/557] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:17.378711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.394722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 281/558] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:17.400301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.480606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e786119e-0580-42d9-ad95-9dccb3daa105 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.483653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:17.516240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163b7fa-8a83-4945-9c6f-66b13ec98bbf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.517676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.518650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.519086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.519562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.530204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.531263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:17.532543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.553499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:17.553971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:17.555002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:17.556129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:17.556129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:17.556129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:17.556129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:17.556346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.556346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.556346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.560874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 278/559] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:17.562007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.568958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.570702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:17.594643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:17.596298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:17.597252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:17.600510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 282/560] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:17.601944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.681234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bc544-8911-44fb-a1a4-03ef07e5aaf1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.726837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.727677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.727965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.728301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.728584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844e32e-71d8-4c0d-bc24-d9dfd2799a21 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.740845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:17.752066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:17.753034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:17.753856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:17.758733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 279/561] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:17.759687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.763953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:17.780085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.781298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.782098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.782958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.784065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:17.784580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.786694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.804132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 283/562] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:17.805991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.855619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:17.888350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f98e8c3-90cb-4dab-b133-e7006b66a00a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.930789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.931713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.932088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.932203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6c62b3-bfa6-418d-b8e7-17f2c9615707 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:17.933100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.934003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:17.934417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.936383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.948996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 280/563] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:17.950241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:17.976789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:17.977745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.978038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.978313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.979060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:17.979995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:17.981311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455802d5-2f4f-4b14-afa5-79e99dd66166 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:17.989691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 284/564] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:17.990999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d14cb-34c9-4958-aec9-2d3bc072a70a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:17.991323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d14cb-34c9-4958-aec9-2d3bc072a70a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:17.991607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d14cb-34c9-4958-aec9-2d3bc072a70a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:17.992984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 285/565] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:17.993565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d14cb-34c9-4958-aec9-2d3bc072a70a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.029598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.071277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cb4f8-c0da-4a2e-86a1-5773876637a1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.094661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.102443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d14cb-34c9-4958-aec9-2d3bc072a70a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.109677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.110925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.111243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.111531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.119853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.121910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5df4782a-1b05-47c3-833b-c8f12c8a941e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:18.136270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:18.137250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:18.138775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:18.139354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.139456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 281/566] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:18.139526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.139526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.139526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.139699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bb6e9-ca4f-4c83-90d1-03c164972140 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.139699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bb6e9-ca4f-4c83-90d1-03c164972140 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.139699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4bb6e9-ca4f-4c83-90d1-03c164972140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.139880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=a7ac2b5986a74ea9923fa0593e58a6b7) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:18.139880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.141322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.159281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 286/567] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:17 2022] GET /identity/v3/role_assignments?user.id=a7ac2b5986a74ea9923fa0593e58a6b7&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:18.159281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.167753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1a4bb6e9-ca4f-4c83-90d1-03c164972140 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:18.169121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a4bb6e9-ca4f-4c83-90d1-03c164972140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:18.244843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 282/568] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:18.247713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a66335-0a67-43e4-91f9-a97db334d5d7 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.248087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a66335-0a67-43e4-91f9-a97db334d5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.248466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a66335-0a67-43e4-91f9-a97db334d5d7 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.250143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 283/569] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:18.250760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a66335-0a67-43e4-91f9-a97db334d5d7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.275984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015bd99-9625-4cad-918a-0e13635d6b29 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.313486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.314291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.314677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.314965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.325840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.328043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.335791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-083abb47-d823-4f1a-823d-08aa059697c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:18.349285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:18.350528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:18.351458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:18.351458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:18.351458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:18.351458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:18.351458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 287/570] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:18.359304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.359304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.359304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.371664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a66335-0a67-43e4-91f9-a97db334d5d7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.375711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:18.376927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:18.384940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.408693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.409369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.409655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.409937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.410773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:18.411085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.412567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.431058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 284/571] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:18.431913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.463276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 288/572] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:18.463276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.556892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee385dfc-8e4c-435a-8a06-f33ca19da790 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.567478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab0f166-6357-4b08-b35f-14e5c840a324 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.569443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.596691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.597497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.597780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.598058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.608515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:18.609565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.610387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.610819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.611300 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.612174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:18.612628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.614144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:18.616261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:18.617382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 285/573] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:18.618498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.639585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 289/574] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:18.640567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.643302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.726131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05728cab-fcb2-4419-a68f-1bf342dcbc82 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.747954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce8cb5-fa1b-4723-b703-e9f77f03a002 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.762227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.763630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.764130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.764585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.777244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.778984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.783718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.784540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.784822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.785093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.795072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:18.796325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:18.797182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:18.798070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:18.798070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:18.798070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:18.798070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:18.798070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 286/575] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:18.798728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:18.807275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:18.808417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 290/576] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:18.809385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.832769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.880270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:18.910559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e9fb26-f236-4166-9676-4e96e8fd1862 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.922365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac201f-a13d-4a4e-aaee-6da44c36aaf3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:18.945072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.945744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.946035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.946321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.947068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:18.947408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.949330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.957553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 287/577] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:18.957898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:18.958330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:18.959088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:18.959400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:18.959673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:18.969553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:18.971418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:18.987015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:18.987974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:18.988916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 291/578] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:18.990098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.055281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.072043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d154ecc8-d1a7-4bf3-a8a7-4dad887d4c7f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.106691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31835b-bcb3-40e5-8f81-abb3ea0bb1f7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.107661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.108326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.108611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.109583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.109583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:19.110214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.111237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.127432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 288/579] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:19.127432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.134970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.147385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.148131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.148412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.148688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.149441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:19.149746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.151113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.164007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 292/580] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:19.164007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.238738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac03df-b3b6-4553-9d8b-54050d7d4199 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.275400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.276720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.277010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.277285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.281867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac951f6b-f6f2-4184-8156-110ca18b58ea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.287750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.289195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.293047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:19.309599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:19.310547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:19.311531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:19.311531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:19.311531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:19.311531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:19.311531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 289/581] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:19.311804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.325886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.326497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.326786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.327063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.328001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:19.328311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.329784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.348137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 293/582] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:19.348924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.393972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.426931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96b9f07-5518-4ac0-b89d-294868c9f35c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.452972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e187099-b805-4f5a-80a2-3bc63a03a2e2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.462430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.463225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.463538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.464369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.471799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.473230 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:19.487428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:19.488315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:19.494177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:19.494629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.494629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.494629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.494629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.494895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 290/583] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:19.494895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.504540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.506002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:19.525740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:19.527235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:19.528311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:19.528311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:19.528311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:19.528311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:19.532035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 294/584] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:19.532720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.555038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.608728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3962aea2-0e1d-4244-89c1-5f3c0c027c2c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.632654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.641791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b398f4-9f51-4963-b99f-7cc72b7d5e76 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.647761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.648430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.648716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.648994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.649718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:19.650027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.651967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.668202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 291/585] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:19.668202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.680508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.681468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.681762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.682041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.693626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.695091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:19.714298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:19.715210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:19.716154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:19.716154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:19.716154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:19.716154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:19.716154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 295/586] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:19.716447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.777601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901e8f8-cf62-4eb4-8d00-0159f174c39c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.796248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.821397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b307e65-2562-4336-a3b4-582919ecc792 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.825039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.825736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.826028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.826313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.827114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:19.827453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.828880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.850534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 292/587] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:19.850534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.861788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.862769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.863059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.863391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:19.864416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:19.864735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:19.866131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:19.878248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 296/588] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:19.879053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:19.890171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:19.961585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e0239-d2b7-40ae-8a03-2a452efe924b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.985634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8758db3d-7c11-4e76-b60e-4cc610961ee9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:19.997976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:19.998778 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:19.999071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:19.999381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.007061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.008534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.021205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.021874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.022162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.022442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.023168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:20.023513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:20.024110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:20.024998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 05:49:20.026045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:20.027946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.029960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 293/589] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:20.030786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.041683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 297/590] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:20.041683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.071131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.105809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.142911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8b309f-9040-4601-ab19-9bfa4a25bd03 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.143475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9b9f5-f075-4d68-b9eb-fc7a0fd03b7a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.178841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.179834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.180123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.180406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/ec34108c1ae24ef699265e0be32e99cf/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.181579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.182431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.183229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.183542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.190529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.192022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.194991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=ec34108c1ae24ef699265e0be32e99cf, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.196433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:20.207132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:20.208174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 298/592] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:20.209133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.215499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 294/592] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:20 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/ec34108c1ae24ef699265e0be32e99cf/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:20.216526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.322534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.327189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444635a-5000-4a29-8e88-ef3335268801 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.341483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830b94ab-8ccf-42d8-809f-0807b8b032fe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.361798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.373701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.374662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.374662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.374948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.375795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:20.376112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.377564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a6519c-a206-4cb0-9671-14854b76e232 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.385991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.386915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.387344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.387628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.394856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.396260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.396731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 299/593] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:20.398120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9fd1-f4fe-466b-ba05-c1a3266539d0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.398414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9fd1-f4fe-466b-ba05-c1a3266539d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.398692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10e9fd1-f4fe-466b-ba05-c1a3266539d0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.400265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 300/594] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:20.401665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9fd1-f4fe-466b-ba05-c1a3266539d0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-81d89639-0f7d-4359-8b19-212455ea9bc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:20.414970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:20.415952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:20.416866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 295/595] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:20.418239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.418590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.418938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.436086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:20.437288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:20.509802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 296/596] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:20.511344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.518219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10e9fd1-f4fe-466b-ba05-c1a3266539d0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.553824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.554503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.554792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.555091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.555953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:20.556266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.558256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.577618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 301/597] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:20.578276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.606831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.625069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.631921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc81b5-0fad-466f-916d-c2e00f5b8785 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.667324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.667822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.668116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.668558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.679078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.680600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.691082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6d4f0e-1eba-4bc0-8e9d-ee8273687497 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:20.699704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:20.700613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:20.701496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:20.701496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:20.701496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:20.701496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:20.702164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 297/598] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:20.707312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.727098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.727799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.728087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.728367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.729151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:20.730069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.731539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.743473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 302/599] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:20.744269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.833159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d347f69f-e9aa-4f33-886d-d7693dc81732 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.855686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c126c-82fb-4c2b-a893-cb177c388f54 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:20.857360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.863981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:20.873449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.874475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.874769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.875057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.885362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:20.893762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:20.894717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:20.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 298/600] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:20.896273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:20.902233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:20.903285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:20.903753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:20.904187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:20.915569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:20.917311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:20.934690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:20.935809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:20.941592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:20.941592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:20.941592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:20.941592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:20.941592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 303/601] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:20.941592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.015704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea76f923-004a-485f-9894-1ae805a7eeed None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.048549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748452e-1808-41aa-b9d3-22b3e7a9144d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.050607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.051374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.051749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.052074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.052864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:21.053317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.054746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.066516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 299/602] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:21.067413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.093194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.094195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.094487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.095024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/00892de83c5444c7a38b921615b2f5ed/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.109787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=00892de83c5444c7a38b921615b2f5ed, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.111277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.119711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.135317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 304/603] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:20 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/00892de83c5444c7a38b921615b2f5ed/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:21.135317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.137734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.195665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3331d29-ef2d-47ad-a1a9-f692205537ff None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.234814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.235556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.235850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.236316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.237089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:21.237409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.238909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.249229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 300/604] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:21.250521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.251189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77478ca-f071-4203-b0b7-3a8d90754e26 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.289124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.289800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.290142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.291853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.291853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:21.291853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.293836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.305606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 305/605] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:21.306294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.373101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3a777-56f6-4b6c-b84b-5c1134079eb1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.384857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.388404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.419652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.420932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.421322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.421698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.432789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.434888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.437581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7233f092-d989-46bc-a3ec-15ca6b451b60 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:21.453480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:21.454480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:21.455419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:21.459333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 301/606] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:21.460370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.482143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.482952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.483238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.483556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.496277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.497734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:21.517181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:21.518106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:21.519449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:21.519449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:21.519449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:21.519449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:21.519449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 306/607] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:21.523701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.571739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820854b-5d4b-4439-81ee-00d68b53b2f1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.607160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.608470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.608960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.609440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.616575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.618291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:21.633361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:21.634340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:21.635265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:21.636584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc62c81-39f0-4ee6-9c68-c9e28cacd284 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.642215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 302/608] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:21.642215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.655871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.681633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.682846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.683148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.683449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.684248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:21.684561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.686005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.691888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.700258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 307/609] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:21.701179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.790210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdbe39-9709-41ed-af6e-06d16d723c0d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.812117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad3a82-58a7-4770-bdeb-4d80ad99aff5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:21.849057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.850191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.850745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.851168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:21.851422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.852034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:21.852394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:21.852721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:21.852824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:21.853308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.853715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:21.854615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:21.855809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.856289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:21.868814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 308/610] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:21.869750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.876611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 303/611] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:21.876611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:21.913519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:21.989289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:22.000268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abaeec-9353-4dbb-875f-abae43a6acbb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.000744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31edb73f-04a7-443b-b07d-ad0ba8780744 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.039511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.041051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.041603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.042559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.042800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/fcefc528820f4d95b779c8972b46bd0f/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.043618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.043994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.044348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/89c777401892485099d5448c506debb0/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.059949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=89c777401892485099d5448c506debb0, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:22.061507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:22.061969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=fcefc528820f4d95b779c8972b46bd0f, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:22.063976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:22.080505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 309/612] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:21 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/89c777401892485099d5448c506debb0/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:22.081650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:22.082647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 304/613] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:21 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/fcefc528820f4d95b779c8972b46bd0f/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:22.188926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:22.208750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8766020-6ba2-4dfb-99f0-9a9c08a61d96 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.249914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.251022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.251499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.251963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.261471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:22.263187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4c230969-b4ba-4f4d-bf12-41dcab9750a8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:22.279971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:22.281179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:22.282329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:22.284752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 310/614] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:22.290744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:22.334050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de834-5c04-45d8-81dd-80809979b82d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.334566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de834-5c04-45d8-81dd-80809979b82d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.334998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480de834-5c04-45d8-81dd-80809979b82d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.337229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 311/615] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:22.342683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.342996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.343276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.370746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:22.372010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:22.419879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb69471-0bf9-4f53-9278-50b4849ac9a8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.450175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 312/616] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:49:22.473808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.474835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.475136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:22.475636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.476118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.477167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:22.477625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:22.479238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:22.498025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:22.519588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 305/617] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Jun 20 05:49:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:22.520510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:22.567862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:22.605358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd5bc48-3a57-4f2c-abc5-1a4cb80d3d76 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.689860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b697d6-2e49-493a-abd3-ac1b6f0cf98e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.704519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.705341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.705632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.705914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.726842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:22.735490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.736285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.736569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.736853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:22.745907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:22.746860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:22.747749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 313/618] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:22.751483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:22.752941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:22.754180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:22.780874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:22.781839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:22.783150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:22.783150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:22.783150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:22.783150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:22.783150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 306/619] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:22.788199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:22.893928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:22.937040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45661c7-4c11-4796-98d5-6956f40942db None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.982913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:22.983614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:22.983904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:22.984183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:22.984935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:22.985248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:22.986635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:22.996730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74b17a-8dbf-481d-929f-ef72ed627e48 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:22.999401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:23.013112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 314/620] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:23.018681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.072602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.073316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.073606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.073883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.074619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:23.074930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.077211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.096160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 307/621] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:23.105671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.155884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec221f6f-f829-4520-be04-53d729b22684 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.193216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.194431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.195017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.195596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.208841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.210625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:23.233363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:23.234288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:23.235167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:23.235167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:23.235167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:23.235167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:23.237741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 315/622] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:23.243631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.308704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:23.341262 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47818f47-811e-46e8-8c5f-4984541bf622 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.349935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e23d6-e3b7-4b89-8ac9-0767d5e10b0a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.364996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:23.388404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.389427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.389948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.390427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.391361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:23.391867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.393431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.408207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 316/623] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:23.413769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.458071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.459029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.459370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.459653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.467592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.469274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-cbdb84ef-7778-49c7-9a5b-e12a157d2bef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:23.485420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:23.486335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:23.487194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:23.491430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 308/624] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:23.492797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336be63e-d6f8-484e-8499-0b4b5ecca778 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.493087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336be63e-d6f8-484e-8499-0b4b5ecca778 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.493352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336be63e-d6f8-484e-8499-0b4b5ecca778 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.499513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 309/625] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:23.500295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336be63e-d6f8-484e-8499-0b4b5ecca778 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.534978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c05f03-6427-4141-b1cc-90d49fb3a7bd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.548725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:23.576863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.578079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.578563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.579131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.587612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.589651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f3028e39-a7c3-4a27-89af-bbaa5db4e2fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:23.606244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:23.607602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:23.608570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:23.615818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 317/626] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:23.617709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.618230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.618723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.619546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336be63e-d6f8-484e-8499-0b4b5ecca778 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.643618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:23.645269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:23.661197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.662151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.662793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.663492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.664995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:23.665716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.668323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.683995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 310/627] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:23.685097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.715836 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:23.733137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 318/628] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:23.734493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.792581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a66028a-429f-4f81-baa6-2756af707632 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.842775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.844308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.845104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.845596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.847176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:23.848003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.850298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.856906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:23.859963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b4695-8419-4f39-954c-4c41d1dea47e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:23.878903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 311/629] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:23.880160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:23.901936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:23.902826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:23.903105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:23.903421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/a7ac2b5986a74ea9923fa0593e58a6b7/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:23.919727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=a7ac2b5986a74ea9923fa0593e58a6b7, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:23.921444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:23.935960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 319/630] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:23 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/a7ac2b5986a74ea9923fa0593e58a6b7/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:23.937138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.016973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.040727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944ff2c-9437-482b-a102-0851ee3b8304 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.052982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b612d-53c6-4ba4-8215-5aeeb2dcae60 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.093569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.094892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.095414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.095891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.117155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.128775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.129962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.129962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.130254 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.151180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:24.167994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:24.169036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:24.169918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 312/631] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:24.175229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.178497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:24.180444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:24.310422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 320/632] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:24.316283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.352875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc4f13-5799-427e-8115-ca3427b90274 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.391956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.418311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.418999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.419326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.419612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.420427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:24.420685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:24.422057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a047f3d0-b4fb-4522-a7bc-76a55d42d31d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:24.431442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 313/633] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:24.433178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ddc20-1364-43eb-95ba-fbf03dfed8dc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.433465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ddc20-1364-43eb-95ba-fbf03dfed8dc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.433749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ddc20-1364-43eb-95ba-fbf03dfed8dc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.439184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 314/634] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:24.440034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ddc20-1364-43eb-95ba-fbf03dfed8dc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.451781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f36b07-d715-40c3-9347-0f94d2389f04 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.454103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.495711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.497430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.497864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.498329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.512378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:24.514929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-faf3be06-2655-47d7-aa32-224511853801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:24.535123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:24.536700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:24.537760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:24.537760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:24.537760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:24.537760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:24.537760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 321/635] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:24.538029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608214e5-0e78-4304-9af7-59005631226e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.542913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608214e5-0e78-4304-9af7-59005631226e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.542913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608214e5-0e78-4304-9af7-59005631226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.561081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-608214e5-0e78-4304-9af7-59005631226e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:24.562741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-608214e5-0e78-4304-9af7-59005631226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:24.564350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ddc20-1364-43eb-95ba-fbf03dfed8dc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.605703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.606790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.607219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.607660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.619555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:24.621812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:24.639796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:24.640723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:24.641583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:24.641583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:24.641583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:24.641583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:24.643217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 315/636] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:24.644151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.650276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 322/637] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:24.652695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26ecc-b486-4006-a87d-f6f96fe2b5dd None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.652990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26ecc-b486-4006-a87d-f6f96fe2b5dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.653272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26ecc-b486-4006-a87d-f6f96fe2b5dd None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.654841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 323/638] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:24.655567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26ecc-b486-4006-a87d-f6f96fe2b5dd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.695194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.721042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.763193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae91b1f-b6a5-49aa-8cbe-2daec663ee17 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.770708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26ecc-b486-4006-a87d-f6f96fe2b5dd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.801680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.802579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.803014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.803493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.804369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:24.804825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:24.805088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.805739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.806020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.806292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.806993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:24.807157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:24.807874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:24.809254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bad703d-01ff-4949-a8df-c5141316960f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:24.824292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 324/639] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:24.825656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.825886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.826165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:24.830857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 316/640] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:24.832265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.846455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:24.848863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:24.922711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 325/641] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:24.923914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:24.947601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.950686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba5c95-c2ae-4279-9099-4c401e4dcb75 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:24.974851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:24.989512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:24.990340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:24.990630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:24.990908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.001079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:25.008760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:25.009687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:25.010539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 317/642] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:25.011229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.038977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f86197-335c-4fde-9b3b-4d02ccf66d47 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.073895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.074690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.074977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.075262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.082187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.083654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:25.099210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:25.100235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:25.101215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 326/643] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:25.101819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.122045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28bcd90-4ff2-4280-9f46-62d017dad23a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.158237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.159165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.159493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.159770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.166476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.168544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:25.183022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:25.184106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:25.185117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 318/644] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:25.185686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.188860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.208896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab973e4-58c9-4b67-9245-2a38428c7efe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.234884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.246829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.247769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.248066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.248346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.259307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:25.267386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:25.268314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 327/645] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:25.274017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.299504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf2c62-9ab1-4f22-a70d-d275c65eb370 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.336147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.337203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.337696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.338220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.339852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:25.340369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.341964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.350841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 319/646] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:25.352006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.388286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdcbd40-c402-4287-9007-5e35491a0fea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.425145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.425835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.426121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.426406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.427270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:25.427605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.429050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.443125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.453455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 328/647] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:25.454738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.461559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23210b1-f3be-4bea-befc-29f8b8870c3e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.464461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.496598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.497266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.497563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.497842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.498624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:25.498938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.500401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.512746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 320/648] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:25.513527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.577826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05d83a-d76c-4965-9b73-68f889c8a871 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.614418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7423c38-85d3-4fe4-b811-f79fa982a7cd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.615930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.616522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.616808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.617647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.617842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:25.618137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.619594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.627915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 329/649] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:25.628540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.658842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.659671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.659967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.660242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.670643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.672124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.691029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:25.691667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:25.692535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:25.693430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:25.693430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:25.693430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:25.693430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:25.693430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 321/650] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:25.693699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.714457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.740044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9da4bae-0d2e-4e34-9c0f-30b2fe00ee97 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.777565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.778365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.778746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.779125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.803893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e952009-d952-46bd-a9cf-db35bbe0f568 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:25.844119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:25.845020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:25.845306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:25.845591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:25.846420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=ec34108c1ae24ef699265e0be32e99cf) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:25.846736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.848213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.855284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 322/651] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/role_assignments?user.id=ec34108c1ae24ef699265e0be32e99cf&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:25.856125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:25.864222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:25.866861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:25.952796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.960633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:25.979722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c8617-c21e-411e-ade8-464937af4af5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.020085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.024293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 330/652] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:26.024293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.025537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.026377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.026658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.038951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.040460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:26.062796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:26.063793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:26.064699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:26.064699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:26.064699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:26.064699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:26.065928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 323/653] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:26.066880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.148359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c26f54-4667-46e2-989f-94913b187f5f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.184674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.185649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.186116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.186569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.187545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:26.188133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.189688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.192724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8865683-4ab3-4e65-ade4-abd89bb43f91 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.202651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 331/654] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:26.203777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.235790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.236330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.242893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.243912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.244192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.244510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.258003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.259749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:26.278545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:26.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:26.280805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:26.280805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:26.280805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:26.280805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:26.283412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 324/655] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:26.284364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.316399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bda119-65f0-4909-8478-69c86cfe8c9f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.355339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.356262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.356547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.356969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.357696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:26.358247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.360233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ecb537-3d2b-45a4-a9bf-fbeee6221d71 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.377968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 332/656] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:26.377968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d844e-df99-40f4-9e32-1928a07d838c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.377968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d844e-df99-40f4-9e32-1928a07d838c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.377968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d844e-df99-40f4-9e32-1928a07d838c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.377968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 333/657] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:26.377968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d844e-df99-40f4-9e32-1928a07d838c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.398847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11287c85-6526-43bc-80b4-8f4fc3cd7ace None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.433606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.434474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.434723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.434994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.442141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.443597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:26.458510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:26.459550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 325/658] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:26.460488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.469347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.489962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d844e-df99-40f4-9e32-1928a07d838c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.505357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.527297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.527940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.528252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.528524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.529227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:26.529519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.530985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83159f5-5792-463a-947c-eaf3fc305724 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.547038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 334/659] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:26.547038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.547038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.547038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.561848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:26.563032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:26.566098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a690eb-6a41-4cd7-b5f1-04dabcaded29 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.601112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.601910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.602195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.602476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.610093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.611585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:26.628011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:26.628966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:26.629897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 326/660] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:26.630487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.637857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 335/661] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:26.640205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.716205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.747459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eb28c8-0c86-4eb8-a59f-53106e58dab2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.748856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69d00e-7d1f-432e-968d-bf5857756291 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.751987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.785905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.786586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.786926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.787151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.787562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.788022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:26.788336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.788431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.788622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.788901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.789775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.797567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.800187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:26.810997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 327/662] 10.222.0.45 () {56 vars in 1137 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:26.811858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:26.823762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:26.824657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 336/663] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:26.830440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:26.938270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ef20-d3ac-4706-81dd-ed2078f2d60c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.945743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e018e10-941f-4690-9371-7deaacb8b1cd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:26.968019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:26.973969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.974771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.975052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.975348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.987016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:26.991189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:26.991884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:26.992163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:26.992437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:26.993299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:26.993578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:26.995656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:26.996867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:26.998030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:27.000341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.000545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 337/664] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:27.000545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.022429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 328/665] 10.222.0.45 () {56 vars in 1139 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:27.022429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.043587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.107604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5de397-0eca-4655-bbcb-807f3a29cbea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.136341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05438bf-04c6-4907-8585-9dd63856b12c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.147008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.148203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.148625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.148909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.174380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.175042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.175369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.175651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.176436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:27.176773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.178179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.190996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 329/666] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:27.191843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.215637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.227932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.229966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.278492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.298671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a7364-010e-407f-9774-08ba2f6a15bb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.333040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.333718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.334009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.334295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.335041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:27.335377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.336802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.344926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 338/667] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:27.344926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.352728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 330/668] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:27.352728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.439289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.448521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83237086-5a27-47d3-95a9-de2746ea96d9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.462573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e03e1a-1e08-43cf-b7cd-aa0003ab6c57 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.485630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.486281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.486557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.486841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.498786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.499476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.499922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.500049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.500844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=00892de83c5444c7a38b921615b2f5ed) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:27.501161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.502581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.508721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 331/669] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/role_assignments?user.id=00892de83c5444c7a38b921615b2f5ed&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:27.520314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.533617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.567270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.569262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.655077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91791124-342c-4a0e-9647-18be2394e6d3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.692440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.703212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.704159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.704514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.704858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.710791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 339/670] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:27.711919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.718291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.720309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:27.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:27.741916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:27.742808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:27.742808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:27.742808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:27.742808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:27.747575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 332/671] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:27.747575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.808053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:27.849967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d2503-16c9-4094-b902-f60844f35eac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.860003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2a5ef4-c135-4074-aa07-dd8149c4181c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:27.890532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.891478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.891835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.892185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.903381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:27.904015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.904327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:27.904680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:27.905024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:27.906006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.918503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:27.920078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:27.924965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:27.926043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:27.927279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:27.927279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:27.927279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:27.927279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:27.928879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 340/672] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:27.929057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:27.943485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:27.945642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:27.948135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:27.948135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:27.948135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:27.948135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:27.948135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 333/673] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:27.948135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:27.985527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.052237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fda280-66ab-4cd1-af08-ce7eda9a2247 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.077107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2eb0b-9c6a-48af-992d-df50553449dd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.088151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.099626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.100288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.100571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.100843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.101560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:28.101863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.103229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.115124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 341/674] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:28.116052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.116771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.117057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.117330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.118050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:28.118357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.120420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3c4a0-08cf-42c2-b3c8-281ec6e9e5f6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.120596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.138609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 334/675] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:28.138609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5066c0-0544-4d8a-82e1-91dcd5420fd6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.138609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5066c0-0544-4d8a-82e1-91dcd5420fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.138609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5066c0-0544-4d8a-82e1-91dcd5420fd6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.138609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 335/676] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:28.138609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5066c0-0544-4d8a-82e1-91dcd5420fd6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.234258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ae296-7fa2-42bf-a5ad-e0c1f48800f2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.237057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.247947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5066c0-0544-4d8a-82e1-91dcd5420fd6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.275138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.276223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.276780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.277113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.277993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:28.278372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.280075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.287352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.288267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.288648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.288929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.295559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 342/677] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:28.296678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.298129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.300230 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-efd5fd4a-98f8-45b3-8cf0-cf4a95285cb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:28.317724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:28.318660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:28.319611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 336/678] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:28.320588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.320948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.321293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.342018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:28.343412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:28.373753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.412271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4ec7d6-bd2e-4a5a-9029-f87035669c4b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.420329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 337/679] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:28.428799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.457099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.458368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.458907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.459258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.467733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.470021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:28.487533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:28.488494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:28.489443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 343/680] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:28.491074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.520892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.540277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0ee09-f1fa-441a-a107-d041c7a92a3e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.573625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.574451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.574717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.574995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.581508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.582948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:28.599653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:28.600906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:28.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:28.605166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 338/681] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:28.606259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.618216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdeb165-92b8-4460-8ffc-4fe16b665f83 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.633215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.663175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.663991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.664336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.664775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.665728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:28.666092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.668781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.687941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 344/682] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:28.687941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.714393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3277b-772d-47ca-9981-f89d1123154c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.751403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.752337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.752619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.752924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.763210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:28.770838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:28.771787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 339/683] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:28.776991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.788534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.806580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae94f3b-71eb-43dc-a20a-7bdc535ce96f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.845206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.846767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.847519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.848447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.851443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:28.852436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.856611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.865238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:28.876292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 345/684] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:28.876936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.891617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa9530-a134-4e17-9037-3a74afe5a499 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:28.929005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:28.929934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:28.930410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:28.930928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:28.931949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:28.932418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:28.934234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:28.947190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 340/685] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:28.948257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:28.982690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce35dd-f305-43c4-924d-d30b657c4fbd None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.016934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.017585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.017871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.018146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.018963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=89c777401892485099d5448c506debb0) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:29.019304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.020740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.027100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 346/686] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/role_assignments?user.id=89c777401892485099d5448c506debb0&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:29.027905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.036835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.071564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c26738e-1cf7-4319-8a70-082f75269c22 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.115407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.117239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.118048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.118537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.120222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:29.120739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.123352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.135742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 341/687] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:29.137389 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.141515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.163086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc97a50-5703-43ee-9e8b-03b05690877d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.202592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.203267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.203558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.203890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.277669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f2caa-a156-4422-9266-7cada4d97018 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.288117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.289641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.299960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.333082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.334416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.334934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.336373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.337428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=fcefc528820f4d95b779c8972b46bd0f) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:29.337934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.339593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.349108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 342/688] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/role_assignments?user.id=fcefc528820f4d95b779c8972b46bd0f&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:29.350403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.408935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 347/689] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:29.414392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.483720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.523774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5131df4b-1975-49fe-ab9c-94a32619e488 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.532600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.568781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b1938-07f1-41b9-bc4d-0219edef0175 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.573187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.574073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.574429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.574780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.586320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.588163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.604947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.605754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.606040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.606322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:29.608490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:29.609361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:29.610697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:29.610697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:29.610697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:29.610697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:29.610697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 348/690] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:29.615620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.617588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.619100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:29.640131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:29.642112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:29.643244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:29.643244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:29.643244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:29.643244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:29.643244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 343/691] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:29.648677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.728996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab8fcc2-e12a-4a65-a512-66527a1554a0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.769371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff5b89c-ac86-4dcc-9855-f264a2b0ed87 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.778305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.778981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.779295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.779583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.780320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:29.780644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.782021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0ec7b7-10b2-4426-a483-7d794f924e00 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.792536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.794848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 349/692] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:29.796607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7aa7b-aa5d-4f36-abab-0fb25bc06b27 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.796902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7aa7b-aa5d-4f36-abab-0fb25bc06b27 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.797192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b7aa7b-aa5d-4f36-abab-0fb25bc06b27 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.798708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 350/693] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:29.803598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7aa7b-aa5d-4f36-abab-0fb25bc06b27 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.824043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.824043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.824651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.827280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.827280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:29.827280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.828123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95eff8-ce5f-44f5-a0b8-f26af9323603 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:29.831208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:29.843552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 344/694] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:29.848365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.848741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.849102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.867589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:29.868961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:29.942471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b7aa7b-aa5d-4f36-abab-0fb25bc06b27 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:29.947398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 345/695] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:29.948401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:29.983437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:29.984437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:29.984727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:29.985391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:29.995094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:29.997386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:30.014047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:30.015923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 351/696] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:30.018073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.076034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9ceba-9dbe-47b2-9761-2ec865056eb1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.086370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:30.113799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.114747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.115458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.115458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.123194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.124782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.130147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:30.140931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:30.142120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:30.143302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:30.149547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 346/697] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:49:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:30.150584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.158749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd11882-3b3c-4206-b91d-af703b82f058 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.195881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.196967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.197407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.197835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.198644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.200165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:30.215392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:30.216592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:30.217631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:30.224825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 352/698] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:30.225799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.279466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d3a81c-2788-4fd3-b2e0-3482476bd52e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.316682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.317602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.318036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.318937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.319923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:30.320775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.322710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.345694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 347/699] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:30.346509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.348967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6308c38d-392e-4bfc-a77d-0c861acabe86 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.380005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:30.398033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.399236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.399748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.400992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.402497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:30.403278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.405557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.424185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:30.433293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 353/700] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:30.433293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.470209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5802e-5c7b-4204-b1a3-5f0ee951665a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.512581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.513655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.513951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.514233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.514950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:30.515281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.517000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.527908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 348/701] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:30.527908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.546871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa028a-3568-4699-9acb-26ebf96e7a15 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.582258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.582944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.583226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.583557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.584548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=a7ac2b5986a74ea9923fa0593e58a6b7) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:30.584880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.586382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.599280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 354/702] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/role_assignments?user.id=a7ac2b5986a74ea9923fa0593e58a6b7&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:30.599280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:30.685717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:30.700858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d666a-90ea-481b-8323-7927cb774ac5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.732441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:30.778021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.778750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.779044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.779349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.800833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54480921-7c1a-41c7-b950-9d5ebbe923ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:30.863160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:30.863890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:30.864448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:30.864737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:30.865964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:30.866562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.868267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.877571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 355/703] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:30.892499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:30.894021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:30.999757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:31.015934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 349/704] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:31.021237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:31.145356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f924b6d-a0e8-42f1-a6d4-8224a3fd28d7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:31.176000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:31.185363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:31.186172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.186460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.186742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.207286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:31.208862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-ea51c9af-6e70-41e5-8343-368efde42c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:31.229160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:31.230096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:31.230994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:31.230994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:31.230994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:31.230994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:31.230994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 356/705] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:31.235950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:31.392972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0d6de-dc6a-4fc5-9b2d-607241b017d1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.393344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0d6de-dc6a-4fc5-9b2d-607241b017d1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.393520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0d6de-dc6a-4fc5-9b2d-607241b017d1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.394242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4901cc-1944-45fd-ac8b-55c7c7d02797 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:31.395179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 357/706] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:31.404772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.405003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.405286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.433910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:31.436261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:31.452923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:31.453894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.454393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.454881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.455969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:31.456492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:31.458032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:31.482492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 350/707] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:31.487896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:31.561941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:31.569065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 358/708] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:31.613530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:31.613752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ecafc-4654-4426-a788-4a180cef1773 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:31.660025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:31.660757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.661046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.661328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.674883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:31.676371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:31.708212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:31.709588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:31.710471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:31.716456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 351/709] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:31.721762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:31.793790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e73fe8-b613-4614-babd-4a5f7affd224 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:31.805764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:31.835695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:31.836636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.837038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.838684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.850858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:31.853090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-513a46ff-e065-4a50-a560-7aa1813580de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:31.876281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:31.877403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:31.878333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:31.878333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:31.878333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:31.878333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:31.878627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 359/710] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:31.880183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e109b7-dc58-4137-a1e7-a06c9fc91d82 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.880644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e109b7-dc58-4137-a1e7-a06c9fc91d82 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.880993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e109b7-dc58-4137-a1e7-a06c9fc91d82 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.882540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 360/711] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:31.888713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e109b7-dc58-4137-a1e7-a06c9fc91d82 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:31.900135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5af5981-bb4f-4088-af8d-1c6080719b22 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:31.914644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:31.949255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:31.950697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.951139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.951637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.953038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:31.953949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:31.955859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596f9a2-2d40-44d4-b7a3-94d9db1eee42 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:31.973274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 352/712] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:31.975498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:31.976185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:31.976640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:31.998805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:32.000605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:32.015882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e109b7-dc58-4137-a1e7-a06c9fc91d82 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:32.052650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:32.053455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:32.053842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:32.054221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:32.055048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:32.055488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:32.057053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:32.075371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 361/713] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:32.076897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:32.091892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 353/714] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:32.093002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:32.120952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:32.212509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7796dc0c-0909-4e78-b610-c423798aaf8f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:32.215484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:32.229907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6a0c1-92c5-4866-864c-9497f9db45f5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:32.255325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:32.256376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:32.256942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:32.257473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:32.258428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:32.260791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:32.273214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:32.274238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:32.274538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:32.274940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:32.276007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=ec34108c1ae24ef699265e0be32e99cf) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:32.276315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:32.278408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:32.291553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 362/715] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:32 2022] GET /identity/v3/role_assignments?user.id=ec34108c1ae24ef699265e0be32e99cf&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:32.291553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:32.292412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 354/716] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:49:32 2022] POST /identity/v3/users => generated 254 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:32.386648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:32.413677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef3bef-07ba-467e-8d68-b2342209c541 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:32.467031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:32.467789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:32.468078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:32.468366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] PATH_INFO: `/v3/services/network` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:32.469023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:32.470678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:32.488857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:32.489969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:32.490832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 363/717] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:32 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:32.498951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:32.610265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8aa339-b6a2-4c22-acb2-22bd0d1ae26a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:32.645658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:32.646332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:32.646619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:32.646897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:32.647641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] RBAC: Adding query filter params (name=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:32.647948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:32.649313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d9efc-7362-43cf-b801-37f623dd496c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:32.655370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 355/718] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:32.661619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:32.809572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a3868-42dc-4eb0-a979-12cdace91d6c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:32.858758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:32.859463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:32.859757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:32.860043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:32.860760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] RBAC: Adding query filter params (type=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:32.861071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:32.862621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268cc88-5d67-406e-9bb5-7aaf4aae7a4f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:32.868184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 364/719] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:33.131965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1fe8d-234a-44e3-9f97-74b1f3b585ab None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.132458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1fe8d-234a-44e3-9f97-74b1f3b585ab None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.132701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec1fe8d-234a-44e3-9f97-74b1f3b585ab None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.134420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 356/720] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:33.139951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7d24-287b-4526-a0f4-0c517b435d04 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.140353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7d24-287b-4526-a0f4-0c517b435d04 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.140724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7d24-287b-4526-a0f4-0c517b435d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.159184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-32db7d24-287b-4526-a0f4-0c517b435d04 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:33.160409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-32db7d24-287b-4526-a0f4-0c517b435d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:33.171172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:33.237613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 365/721] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:33.282661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1fe8d-234a-44e3-9f97-74b1f3b585ab None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:33.320986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:33.396318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdcf64-cc6e-45fc-a0cb-3751c257638a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.396641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdcf64-cc6e-45fc-a0cb-3751c257638a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.396927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fdcf64-cc6e-45fc-a0cb-3751c257638a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.399738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 366/722] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:33.404108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.404408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.404689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.421962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:33.423163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:33.432668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec1fe8d-234a-44e3-9f97-74b1f3b585ab None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:33.499404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 367/723] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 20 05:49:33.515618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:33.515929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:33.516801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.517089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.517412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.544158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:33.551780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:33.552760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:33.553631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:33.559694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 357/724] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:33.565369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:33.599646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:33.618776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f33d3c-86ac-4c39-8350-b144041ea3ad None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:33.657935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d900e-355e-419d-be60-764443657082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:33.658524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d900e-355e-419d-be60-764443657082 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.658809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d900e-355e-419d-be60-764443657082 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.659083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d900e-355e-419d-be60-764443657082 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.659831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d900e-355e-419d-be60-764443657082 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:33.661198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d900e-355e-419d-be60-764443657082 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:33.692696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 368/725] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:49:33 2022] POST /identity/v3/services => generated 251 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:33.709407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:33.733216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4772755f-8e46-4357-b560-39935d0e16b9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:33.788893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:33.789923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.790390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.790895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.791861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:33.792386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:33.793909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:33.806380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 358/726] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:33.811826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d900e-355e-419d-be60-764443657082 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:33.952108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d900e-355e-419d-be60-764443657082 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:33.990863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:33.991696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:33.991981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:33.992258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:33.994265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:34.005350 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:34.006808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:34.023785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:34.024708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:34.026137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:34.026137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:34.026137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:34.026137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:34.026137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 369/727] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:34.032727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:34.138101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0eeec-928c-48c5-b300-e672804f05bb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:34.176575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:34.177251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:34.177534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:34.177811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:34.178534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:34.178842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:34.180222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:34.193138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:34.196632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 359/728] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:34.202929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:34.361863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2595810d-3588-44c9-ad65-da8f69b2c426 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:34.412098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:34.412900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:34.413189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:34.413464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:34.426291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:34.427809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:34.448651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:34.449610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:34.450573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 370/729] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:34.457288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:34.579186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eda42f-44d0-4f90-8d67-faf04a024bdd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:34.614711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:34.615430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:34.615719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:34.615990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:34.616825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:34.617133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:34.618545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c0374e-5b18-49ba-8f84-ca50db712ddf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:34.637537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 360/730] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Jun 20 05:49:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:34.643204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:34.720015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:34.785247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b537929c-0bce-4137-aa42-355758aa9f16 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:34.825546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:34.826282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:34.826492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:34.826775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:34.937501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:34.938986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:35.007782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.013173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23e37-ecf9-4e66-bbc5-0285ba06728f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.013479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23e37-ecf9-4e66-bbc5-0285ba06728f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.013768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d23e37-ecf9-4e66-bbc5-0285ba06728f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.015420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 361/731] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:35.032741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ad48b-36ba-4724-b71b-956edca53a71 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.033037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ad48b-36ba-4724-b71b-956edca53a71 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.033325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ad48b-36ba-4724-b71b-956edca53a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.069730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ce7ad48b-36ba-4724-b71b-956edca53a71 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:35.070943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ce7ad48b-36ba-4724-b71b-956edca53a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:35.079983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 371/732] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:34 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 20 05:49:35.085610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.199800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.223571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 362/733] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:35.224986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e06ed-64b0-4224-b5d1-c725a1a2cf9c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.225285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e06ed-64b0-4224-b5d1-c725a1a2cf9c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.225570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e06ed-64b0-4224-b5d1-c725a1a2cf9c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.231725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 363/734] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:35.232906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.233199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.233485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.250934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:35.252388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:35.262037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f55542-51f1-4863-99f3-6a282166f4e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.307967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.308872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.309157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.309430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.320855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:35.322392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:35.328890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 364/735] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:35.329903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.330156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:35.342047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:35.343002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:35.344203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:35.344203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:35.344203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:35.344203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:35.348197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 372/736] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:35.349529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.439141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a88339e-4181-472c-b7c3-bace8a628ca1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.457443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.457731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d70f8-5839-48a9-ad7b-2a2acc0b8091 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.476351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.477285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.477647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.477998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.488709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:35.494402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.495063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.495391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.495681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.496521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:35.496682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:35.497548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:35.499029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:35.499705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:35.499789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 365/737] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:35.499789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.502778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17009ea5-7714-4698-a59d-3484249c4067 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:35.518649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 373/738] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:35.518649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05f15-4f58-4abd-8bcb-8bd080d81c9f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.518649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05f15-4f58-4abd-8bcb-8bd080d81c9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.518649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05f15-4f58-4abd-8bcb-8bd080d81c9f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.518649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 374/739] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:35.518649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe05f15-4f58-4abd-8bcb-8bd080d81c9f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.583409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.607118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526df3a-badd-4430-9150-a585b9204d11 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.632620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe05f15-4f58-4abd-8bcb-8bd080d81c9f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.640156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.640966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.641254 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.641532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.652848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:35.660427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:35.661376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:35.662215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 366/740] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:35.662901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.672301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.672972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.673257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.673529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.674267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:35.674573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:35.675989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc826a2-1850-4be6-8a17-bc6ea0cf2a56 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:35.684915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 375/741] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:35.686072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.686365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.686644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.703179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:35.704409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:35.708035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.779003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867384d9-0c85-471c-8000-24f87f1841c1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.783399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 376/742] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 20 05:49:35.784382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.821002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.822197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.822671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.823293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.831392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:35.833049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:35.847646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:35.848733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 367/743] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:35.849644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.857648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.907851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f5efc-465e-4025-89f2-85b1fa21c976 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.943947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.945341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.945836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.946321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:35.947802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:35.948303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:35.949682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:35.953747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:35.963450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57b889c-5eac-49b0-a1a4-29fe94f3a174 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:35.963612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 377/744] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:35.963612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:35.997888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:35.998689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:35.998977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:35.999285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.010090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.011573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:36.029103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:36.029984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:36.030889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:36.030889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:36.030889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:36.030889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:36.030889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 368/745] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:36.031171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.071884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e3e7e-1c8c-4e3f-abe7-863c00a8526b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.085503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:36.110902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a076-7876-4221-9232-5d0138969968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.111597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a076-7876-4221-9232-5d0138969968 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.111885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a076-7876-4221-9232-5d0138969968 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.112284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731a076-7876-4221-9232-5d0138969968 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.113004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a076-7876-4221-9232-5d0138969968 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.114493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731a076-7876-4221-9232-5d0138969968 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.128471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 378/746] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:49:35 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:36.129308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a076-7876-4221-9232-5d0138969968 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.151124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938a908-ef94-48b9-8cf7-e52ae0a50429 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.197839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.199156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.199653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.200147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.201758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:36.202302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.204043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.218837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 369/747] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:36.220704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.232655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:36.266599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731a076-7876-4221-9232-5d0138969968 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.305024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.305763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.306047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.306863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.318387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.319901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:36.340193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:36.341166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:36.342248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:36.342248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:36.342248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:36.342248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:36.342248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 379/748] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:36.342622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.351367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012b4ae-056b-46e8-9829-7ede819a4c2e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.369659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:36.389108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.389798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.390069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.390349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.391081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:36.391422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.392791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1f65d-7a3f-4551-aa85-1e0afb75fc5c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.405882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 370/749] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:36.406986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb308-c650-46ed-9099-c24bcbf93de9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.407330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb308-c650-46ed-9099-c24bcbf93de9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.407613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3eb308-c650-46ed-9099-c24bcbf93de9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.411647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 371/750] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:36.411647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3eb308-c650-46ed-9099-c24bcbf93de9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.455523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d99e55-622a-40dd-acbf-a09018573236 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.483656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:36.492943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.493742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.494655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.495042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.496079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=a7ac2b5986a74ea9923fa0593e58a6b7) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:36.496498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.498108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.510490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 380/751] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/role_assignments?user.id=a7ac2b5986a74ea9923fa0593e58a6b7&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:36.511932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.524225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3eb308-c650-46ed-9099-c24bcbf93de9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.571321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.571961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.572261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.572548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.573743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:36.574083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.575872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1191a5cf-cc2f-482c-bd5b-b59d019d731b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.592988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 372/752] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:36.595206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.595532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.595831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.618323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:36.620205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:36.639616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:36.657542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbfeac-355e-4184-a6a0-71c3b4928296 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.696201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.697480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.697769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.698050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.707450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.707720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 373/753] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:36.707984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.709212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:36.724849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:36.725822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:36.726803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 381/754] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:36.736352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.800474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:36.866450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9cf2d9-3735-4e9f-8ea2-c5e07f1e8912 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.885832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05648cb6-3ba9-4d7a-b236-f828b19ddcb6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:36.904251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.905055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.905344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.905626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.913329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.914781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:36.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:36.931960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 374/755] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:36.933308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.934425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:36.935391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:36.935883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:36.936385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:36.937436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:36.938052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:36.939811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:36.970341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 382/756] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:36.970341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:36.979744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.073330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbaf8d4-6838-4780-896a-281347430bfc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.085825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25da9a5f-ccba-4c56-8a7b-0ea7b1177d17 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.094859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.117588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.118538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.119378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.119661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.120569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.122591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.126727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.127670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.128162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.128658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.129716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:37.130249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.131998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:37.138468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:37.139720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:37.140594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:37.144789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 375/757] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:37.145399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.154505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 383/758] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:49:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:37.156387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.233542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.262506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41be6667-b079-4f8c-8d27-036f479cf2f2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.292544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7d17e5-4ca1-4a81-b0fc-ace68ef14111 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.300970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.302156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.302449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.302795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.337146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.338031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.338729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.339012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.340033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:37.340579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.342617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.349183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 384/759] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:37.350798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.368569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.389142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.390693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.467170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61911ae4-b323-4af7-b356-783b60cf7fea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.489292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 376/760] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:49:37.490284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.498554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.508755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.509607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.509896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.510173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.521550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.523003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:37.542969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:37.543893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:37.544797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:37.544797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:37.544797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:37.544797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:37.548894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 385/761] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:37.549732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.570480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.590556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa64a681-4e21-4fd8-844f-0f28a0059f59 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.625351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.626018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.626303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.626579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.627300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:37.627612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.628962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.640667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 377/762] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:37.641704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.679675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534afc67-b973-414a-ab3b-e0413900131f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.732226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.733180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.733475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.733760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.746869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e46e1f-2912-4fed-bd70-380bce3bd207 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.748063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.750641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:37.775484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:37.777275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:37.778789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:37.778789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:37.778789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:37.778789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:37.778789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 386/763] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:37.778789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.792423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.793436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.793734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.794020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.794830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:37.795409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.797557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.811962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.822896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 378/764] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:37.824136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.829141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:37.887411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e19a22-de5c-4b4d-982a-25b879009601 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.923561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.924612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.924766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.925092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.926042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] RBAC: Adding query filter params (interface=public, service_id=eda81c6d754b4356829c23c49eba981e, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:37.926439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.927984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.939568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 387/765] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/endpoints?service_id=eda81c6d754b4356829c23c49eba981e&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:37.940525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:37.945001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a4991-2558-4a9a-9b68-b451776de355 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:37.982335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:37.983409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:37.983906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:37.984391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:37.985320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:37.985837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:37.987479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:37.999837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 379/766] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:38.000897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.039525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:38.047729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc7423-c12b-4353-93cf-488d75780ce1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.068158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:38.089191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.089996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.090278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.090553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.099677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.101144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.112112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020198a5-3508-4c8a-9752-5853dc7c26ec None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:38.115825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:38.116715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:38.117695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:38.121837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 388/767] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:38.122720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.152803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.153855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.154349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.154831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.155786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.157378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.163324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 380/768] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:38.164368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.236316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4993cc36-4d38-4917-8d92-25712a3ba56c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.273238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.274043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.274328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.274627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.282739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.284285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d7f3ae08-2ef4-4846-b99e-8856642f3eb2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:38.299441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:38.300435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:38.301362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 389/769] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:38.302428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd80d-1528-4893-9d64-2b14ff3f23b8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.302715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd80d-1528-4893-9d64-2b14ff3f23b8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.302993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdfd80d-1528-4893-9d64-2b14ff3f23b8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.308313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 390/770] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:38.308313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd80d-1528-4893-9d64-2b14ff3f23b8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.331070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:38.347496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc217240-e30c-4563-a064-66b73a215e39 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.383066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:38.393503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.394596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.394952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.395168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.396013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:38.396333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.398571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1cb20-fdfd-4f2a-a006-0008e3f42dd6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.412083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 381/771] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:38.413333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.413623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.413905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.430023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:38.431805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:38.431994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdfd80d-1528-4893-9d64-2b14ff3f23b8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.476902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.477621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.477907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.478218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.479437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:38.479555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.481049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.495273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 391/772] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:38.496103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.508601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 382/773] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:38.510232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.610015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:38.618084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b582b6a-8530-4897-a4be-43b046994c28 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.635266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621e27c1-7a7e-4d77-b1df-6daba730f6b5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.637554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:38.657624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.658567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.658962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.659147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.660021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=306d74f4500845a097e8e062af12dc0f) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:38.660346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.661798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.667775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 383/774] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/role_assignments?user.id=306d74f4500845a097e8e062af12dc0f&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:38.668708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.678838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8612ac4-f538-439c-8c39-8793c22ec049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.680136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612ac4-f538-439c-8c39-8793c22ec049 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.680626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612ac4-f538-439c-8c39-8793c22ec049 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.681086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8612ac4-f538-439c-8c39-8793c22ec049 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/ec34108c1ae24ef699265e0be32e99cf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.684013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8612ac4-f538-439c-8c39-8793c22ec049 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=ec34108c1ae24ef699265e0be32e99cf, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.685753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8612ac4-f538-439c-8c39-8793c22ec049 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.709370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 392/775] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:38 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/ec34108c1ae24ef699265e0be32e99cf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:38.709370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e1f1b-dcbd-4e94-964e-0475165c0cac None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.709370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e1f1b-dcbd-4e94-964e-0475165c0cac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.709370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e1f1b-dcbd-4e94-964e-0475165c0cac None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.709370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 393/776] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:38.724431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.724812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.725032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.765612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:38.766922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:38.866669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76e1b3-99cc-4170-85d4-ad7fdc7d9cf2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:38.894730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 394/777] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:38.918298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:38.924620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:38.925748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:38.926242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:38.926737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:38.927671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:38.929281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:38.942736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:38.943803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:38.944651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:38.947624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 384/778] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/services/network => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:38.952907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.002459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:39.004858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:39.051318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb200bd-b7c7-4e08-b7cb-ea2706868fb3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.090815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60e35e5-c8a3-4919-a0cc-f47fcfff3d62 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.117208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.118012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.118298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.118578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.137472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:39.145252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:39.146157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:39.147082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 395/779] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:39.154893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.162316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.163006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.163496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.163599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.164321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:39.164635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.166591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.181343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 385/780] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:39.188202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.276756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe0a765-faca-4f07-a50a-8bc564be5c35 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.311684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:39.315914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.316872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.317386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.317858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.318794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:39.319433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.321265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.332416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 396/781] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:39.339198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.441925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafaa27-1be0-4109-8c42-ae3798ad3ef7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.446074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e568190a-83b6-46a3-bd09-ed830943abe4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.460961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:39.488871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.489955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.490489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.490960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.491949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:39.492626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.494854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.500898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 386/782] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:39.508202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.527342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.528146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.528430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.528706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.544083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.545560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:39.578249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:39.579232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:39.580164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:39.580164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:39.580164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:39.580164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:39.590928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 397/783] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:39.597200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.668629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018a805-a8ff-4716-93c1-ba81a826dda3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.702673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.703274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.703584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.703871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.704668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] RBAC: Adding query filter params (interface=public, service_id=d5fe5a0f04334cd099f2473e2fc59f6b, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:39.705044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.706463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.714452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 387/784] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/endpoints?service_id=d5fe5a0f04334cd099f2473e2fc59f6b&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:39.718725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:39.720389 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.803601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:39.826961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ae7b2-f896-4bab-ac6d-21f5661bfec6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.892539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.893548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.894061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.894545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.895349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75782987-eb18-4a49-9b30-e5863e185163 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:39.895808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:39.896635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.898243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.918911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 398/785] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:39.924447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:39.929364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:39.929514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:39.929715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:39.930003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:39.930730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:39.932669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:39.942942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 388/786] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:40.037641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:40.042108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816bc41c-e68a-4443-858a-6994472e6315 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:40.082408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:40.083629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.084122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.084622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.094666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:40.096643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:40.112367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:40.113364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:40.114302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:40.122863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 399/787] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:40.123927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:40.286975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e7ac8d-b7f2-46b0-bf95-cc5be4bee499 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:40.288568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:40.332240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:40.332918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.333260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.333544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.334386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:40.334699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:40.336139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:40.362932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 389/788] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Jun 20 05:49:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:40.368283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:40.494000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfb7ab-1be6-45c7-a75c-d73f71b341cc None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:40.529121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:40.530289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.530774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.531390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.543117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:40.544824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f09325de-46d2-44ab-91d9-7a33456811c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:40.562785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:40.563811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:40.564993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:40.564993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:40.564993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:40.564993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:40.566862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 400/789] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:40.572639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:40.607962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42278b8-eac1-48e4-82b2-739de1505b84 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.608405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42278b8-eac1-48e4-82b2-739de1505b84 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.608791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42278b8-eac1-48e4-82b2-739de1505b84 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.610511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 401/790] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:40.615579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.615949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.616304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.640125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:40.643628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:40.650808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:40.726827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 402/791] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:40.754189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:40.761419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2796d2-fba1-4659-9277-7b4dfc96b851 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:40.865518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ae8d5-c54a-48d8-8105-3eb10975c7e4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:40.879970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:40.880661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.880949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.881231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.881976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:40.882290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:40.883716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:40.903271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:40.904117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:40.904416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:40.904694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:40.905933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:40.907347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:40.917626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 390/792] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:40.919964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:40.921100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:40.922075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 403/793] 10.222.0.45 () {56 vars in 1129 bytes} [Mon Jun 20 05:49:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:40.926984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:40.927188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:40.955493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:40.990617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.084815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfecfcd-c2e3-4abf-a1b2-9dc089a0dbd9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.117375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff8151-14bf-4335-8cf8-586aefa92b42 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.129817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.130630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.130918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.131203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.144023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:41.146435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:41.163271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.164206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.164666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.165128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.166064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:41.166583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:41.168214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:41.174924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:41.175854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:41.176736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:41.181165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 404/794] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Jun 20 05:49:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:41.182534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 391/795] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:41.186558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.187922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.279491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.289486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.310935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f851081-e9d5-43f9-9f1d-8d0a0f61e014 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.313742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293d298-eb05-4fb9-bb82-5fe04d0a266e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.346178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.346762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.347046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.347355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.348140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:41.348447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:41.349855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:41.351219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.351906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.352192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.352464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.353700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:41.354006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:41.355385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa52dba-3944-4b40-ada0-7e8e781516b5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:41.367802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 405/796] 10.222.0.45 () {56 vars in 1124 bytes} [Mon Jun 20 05:49:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:41.367802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e201c13-5d28-4707-9d8c-2b3f84fa331c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.367802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e201c13-5d28-4707-9d8c-2b3f84fa331c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.367802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e201c13-5d28-4707-9d8c-2b3f84fa331c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.367802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 406/797] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:41.367802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ddbff-8321-4dda-9be9-f8cb2f92c845 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.368274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ddbff-8321-4dda-9be9-f8cb2f92c845 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.368274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ddbff-8321-4dda-9be9-f8cb2f92c845 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.368274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 407/799] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:41.368520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 392/799] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:41.369104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ddbff-8321-4dda-9be9-f8cb2f92c845 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.369181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.472218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9ddca-6b0f-4f38-b871-fb533ad08b0a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.474217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ddbff-8321-4dda-9be9-f8cb2f92c845 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.506458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cae80bb-de42-4b57-9eaf-ee0aef4e0170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.507341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae80bb-de42-4b57-9eaf-ee0aef4e0170 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.507569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b51d-a571-415a-9449-bb8d91a569dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.507687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae80bb-de42-4b57-9eaf-ee0aef4e0170 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.507889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cae80bb-de42-4b57-9eaf-ee0aef4e0170 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/a7ac2b5986a74ea9923fa0593e58a6b7/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.508217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b51d-a571-415a-9449-bb8d91a569dd None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.508499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b51d-a571-415a-9449-bb8d91a569dd None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.508775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b51d-a571-415a-9449-bb8d91a569dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.509583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192b51d-a571-415a-9449-bb8d91a569dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:41.509936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae80bb-de42-4b57-9eaf-ee0aef4e0170 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=a7ac2b5986a74ea9923fa0593e58a6b7, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:41.510956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192b51d-a571-415a-9449-bb8d91a569dd None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:41.511398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cae80bb-de42-4b57-9eaf-ee0aef4e0170 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:41.514442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.522459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.528177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 393/800] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:41 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/a7ac2b5986a74ea9923fa0593e58a6b7/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:41.532626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.532626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.532626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.551744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 408/801] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:49:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:41.553666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.554071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:41.554304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.554730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.555720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:41.590169 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:41.591438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:41.726026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 394/802] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:41.736316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 409/803] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:41.758214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.761943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.874896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f6dca-220f-4b65-a1cb-c3163e358db1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.881256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a421836-8b2a-4ab1-a11c-3a1b0d920b7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:41.902918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.912615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.913092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:41.913550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.913716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.913962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.931581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:41.940603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:41.941548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:41.942375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:41.945162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 395/804] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:41.954012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:41.968413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:41.969519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:41.969821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:41.970093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:41.990006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:42.001713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:42.002673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:42.003528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:42.004770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 410/805] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:42.020303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.094585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ff47c-9662-41fa-af2e-212fafb43e6f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.140589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.143323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.144074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.144697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.145650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:42.146189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.147895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.158172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 396/806] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:42.163815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.217908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:42.242958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219ce31-6407-496b-8692-bbc2f6287023 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.275281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb651d-b85e-4dbe-a7a9-762676d53e2b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.282406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.283080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.283399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.283680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.284500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:42.284741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.286142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.294893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 411/807] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:42.308441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.327909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.329041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.330275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.330888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.354673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.357103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.372902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-271421d8-df30-40ad-b3e1-0dec234e7bf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:42.376657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:42.377537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:42.378900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:42.378900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:42.378900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:42.378900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:42.378900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 397/808] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:42.383633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75e19-e111-4707-8a4c-a1dc383205af None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.383947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75e19-e111-4707-8a4c-a1dc383205af None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.384269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b75e19-e111-4707-8a4c-a1dc383205af None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.389605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 398/809] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:42.389605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75e19-e111-4707-8a4c-a1dc383205af None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.446565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c9a9a-34c6-43a0-9c7c-a6b618e02404 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.485327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.486581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.487101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.487670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.495533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:42.499519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b75e19-e111-4707-8a4c-a1dc383205af None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.503181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.504962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-60224514-98b3-4e47-a5a4-a598a5bda98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:42.524378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:42.525869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:42.526740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:42.526740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:42.526740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:42.526740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:42.529179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 412/810] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:42.530829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.531356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.531897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.539300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01343f1-130a-4848-b463-290680efef17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.539824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01343f1-130a-4848-b463-290680efef17 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.540109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01343f1-130a-4848-b463-290680efef17 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.540946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01343f1-130a-4848-b463-290680efef17 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.541679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01343f1-130a-4848-b463-290680efef17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:42.541989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01343f1-130a-4848-b463-290680efef17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.543377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01343f1-130a-4848-b463-290680efef17 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.552473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:42.553959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:42.560142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 399/811] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:42.561048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01343f1-130a-4848-b463-290680efef17 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.635601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 413/812] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:42.636847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.665145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01343f1-130a-4848-b463-290680efef17 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.682673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:42.701047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.701601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.701984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.702359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.703232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:42.703902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.705167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.717648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 400/813] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:42.718544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.747554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:42.770610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a563a4-14ba-4ef7-b72d-a5f880df50af None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.807725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.808819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.809268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.809738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.826423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.828346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.834010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bac571-6ca5-4ae7-80e3-1b8f6026f976 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:42.845938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:42.846906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:42.847891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:42.850704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 414/814] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:42.851802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.878063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:42.878952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:42.879381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:42.879728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] PATH_INFO: `/v3/services/network` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:42.880593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:42.882161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:42.895602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:42.896616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 20 05:49:42.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:42.902317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 401/815] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:42.903313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:42.957396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:42.975487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4af815-842c-49df-9ed1-10e6afb600c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.000588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.016606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.017656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.018130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.018236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2353622-7bd6-4ce9-bcca-ff8627262dfe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.018707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.026477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.028166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:43.043913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:43.044884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:49:43.045823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:43.048620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 415/816] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:43.049580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.059945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.060695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.061050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.061394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.062403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:43.062799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.064480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.091560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 402/817] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Jun 20 05:49:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:43.092929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.157586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1c6c8d-4c4a-45a8-a905-1bd5de034f74 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.194489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.195576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.196063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.196591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.198129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] RBAC: Adding query filter params (name=network) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:43.198630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.200291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.201896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52c610a-5852-4ba7-9805-979478fce833 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.205857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 416/818] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:43.206893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.218158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.239156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.239868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.240148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.240444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.241704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:43.242023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.244279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.262129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.267420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 403/819] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:43.268499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.322686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90a620-8df3-4980-ad89-d96bb1c1783d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.364763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.365682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.365968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.366247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.381160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.382961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.389036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f548068-4ea6-4d51-a650-98f3310ae085 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:43.406200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:43.407128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:43.408172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:43.408172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:43.408172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:43.408172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:43.411327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 417/820] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:43.412544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.432062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.432725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.433019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.433398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.434169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] RBAC: Adding query filter params (type=network) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:43.434440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.435844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.445527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 404/821] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:43.446423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.495695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.513135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.539894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7001e454-8903-402c-a682-e664a8a1a4cf None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.552770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12e95e-4ec2-4cfc-b0cb-79f1ee3e1a2d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.578063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.579162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.579920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.580519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.588983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.589655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.589945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.590222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.590953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:43.591276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.592694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.610229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 405/822] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:43.610229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.666386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.668628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.712667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb394a-0b24-447a-a1eb-5436fd86d830 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.733172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.736744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:43.749233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.749948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.750183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.750464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.751277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.752596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:43.779432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 406/823] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:49:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:43.780733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.805512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 418/824] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:49:43.807483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:43.934326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03af096-42e3-43f9-8b4d-754b2120b5bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.940973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d24c846-3226-432d-a562-888e6f0c2649 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:43.975944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.976771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.977058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.977336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.981711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:43.982693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:43.983131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:43.983605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:43.992644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:43.992960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-13a96dd9-59b1-4f95-ac5b-14565717e538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:44.008918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:44.010461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:44.011603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 407/825] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:44.012224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd724187-dde5-475b-a2dd-c1d4fefd0a15 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.013092 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd724187-dde5-475b-a2dd-c1d4fefd0a15 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.013092 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd724187-dde5-475b-a2dd-c1d4fefd0a15 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.014294 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 408/826] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:44.015706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02c374-7767-4790-93b5-907f66fca99c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.015997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02c374-7767-4790-93b5-907f66fca99c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.016273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab02c374-7767-4790-93b5-907f66fca99c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.017914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 409/827] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:44.022789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02c374-7767-4790-93b5-907f66fca99c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.061373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.106726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.108192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c86b02f6-8b40-4520-be30-e005b7934809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:44.125599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:44.126675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:44.127750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:44.127750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:44.127750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:44.127750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:44.127750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 419/828] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:44.128170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.128239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.128489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.131877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab02c374-7767-4790-93b5-907f66fca99c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.144385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:44.145564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:44.152810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.165721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.166382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.166663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.166943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.167753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:44.168065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.169473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db7c95-46af-4133-9354-66d943b49af3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.180678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 410/829] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:44.182258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.182550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.182827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.198159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:44.199353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:44.213143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 420/830] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:44.213985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.264600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 411/831] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:44.265705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.293210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.314546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc1a9d-aa47-47d9-adc7-d86da549fc78 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.346946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.347638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.347921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.348197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.348917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:44.349223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.350598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.361904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 421/832] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:44.362649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.364855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd412b2-bed7-46a0-a12b-84e7b69cd301 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.365243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.398750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.400179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.400464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.400737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/306d74f4500845a097e8e062af12dc0f/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.414298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=306d74f4500845a097e8e062af12dc0f, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.415824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.428226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 412/833] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:44 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/306d74f4500845a097e8e062af12dc0f/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:44.428812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.469550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a3e99-59ba-439f-a2e6-924f54318358 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.505903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.507224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.507672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.507954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.515986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.522095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:44.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:44.531975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:49:44.532800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:44.537505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 422/834] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:44.538311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.546908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb11090-1d49-4326-82d0-48aa0bd2a496 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.594365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.595108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.595439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.595765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.596430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.597791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.608848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:44.610456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:44.612268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:44.613107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 413/835] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:44.613902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.654213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ff85fd-10ae-4cc2-9003-61bcf6dce7c8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.688750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.689539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.689822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.690141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.696776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.698221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:44.712709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:44.713610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:44.714530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 423/836] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:44.714998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.730809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f834b-5498-4db8-83a2-406bdee7add9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.768459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.769240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.769529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.769808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.770545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:44.770854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.772258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.786411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 414/837] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:44.786411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.787820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.828175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b28c13e-56ff-469f-84f1-25b4b44320da None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.851887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:44.863785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.864611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.865017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.865291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.865992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:44.866296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.867686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.878396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 424/838] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:44.878396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.902619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f28335-1c7f-4341-b22e-14ea890d952b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:44.936082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:44.936742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:44.937027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:44.937307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:44.938090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:44.938400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:44.939854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:44.959714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 415/839] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:44.959714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:44.988246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9615a8c-ccaf-43f0-bb3f-f04d26832dc1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.022388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.023181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.023513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.023796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.024465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:45.034197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.035688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:45.053490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:45.054407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:45.055338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:45.055338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:45.055338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:45.055338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:45.055338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 425/840] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:45.055629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:45.073299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbf8afb-90d1-45a5-97c2-42d7657fdb92 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.097857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:45.111978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.112576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.112867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.113163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.113864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:45.114175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.115809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.131286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 416/841] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:44 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:45.131286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:45.188316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8ccaf-6bb5-49c4-8322-2ed6bfa34a82 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.237267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.238265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.238750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.239202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.240469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=ec34108c1ae24ef699265e0be32e99cf) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:45.241065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.242698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.250352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 426/842] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:45 2022] GET /identity/v3/role_assignments?user.id=ec34108c1ae24ef699265e0be32e99cf&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:45.258410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee066520-5569-4473-a35a-9fc0574c972c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.289950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:45.299399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.300091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.301434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.301434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.301434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:45.301687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.303095 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.317140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 417/843] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:45.323193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:45.435357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4343337-d4ea-47f7-a718-3801c9a5797b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.437064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:45.473122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.473931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.474219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.474502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.482205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.483701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:45.499266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:45.500247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:45.501708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 427/844] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:49:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:45.506028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:45.614307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cd35a5-340e-44ba-a003-31678a9386a3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.649140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.649859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.650145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.650421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.651281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:45.651661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.653081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.671676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 418/845] 10.222.0.45 () {56 vars in 1135 bytes} [Mon Jun 20 05:49:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:45.677192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:45.778806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1cee9-edee-4e92-bc44-d7fdfe60b5d6 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:45.813228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:45.813882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:45.814168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:45.814442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:45.850313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:45.888584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:45.890036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:45.990944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 428/846] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:49:45.996727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.044456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:46.102510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7da532-be1b-45d5-8efb-a8a6d3e9dd65 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:46.138748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:46.139581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.139855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.140131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.151042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:46.152515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:46.284236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:46.285181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:46.286514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:46.286514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:46.286514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:46.286514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:46.286514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 419/847] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:46.291899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.363713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:46.396751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd170b1d-683c-4809-b5f6-cf671208d9e5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:46.431968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:46.432995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.433485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.433988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.434902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:46.435440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:46.437014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ef89e-7903-44c2-8b14-d93e592e3c55 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:46.449823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 429/848] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:46.455172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.456752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb687b-f957-4819-a36f-e2a56feffbd9 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.457052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb687b-f957-4819-a36f-e2a56feffbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.457340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb687b-f957-4819-a36f-e2a56feffbd9 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.459298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 430/849] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:46.464614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.464912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.465200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.483932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:46.485135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:46.567326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 431/850] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:46.573103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:46.580773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9672b585-af43-47ea-a601-b0316f4b6d4a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:46.582798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.626728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:46.628324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.629249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.630008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.638834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:46.640707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:46.642183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:46.661799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:46.662675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:46.663599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:46.666727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 420/851] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:46.671781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.696975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472ca490-82da-4c54-a540-7087b4eb0320 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:46.739590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:46.740534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.740824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.741102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.753876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:46.763602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:46.764663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:49:46.765504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:46.771302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 432/852] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:46.771302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.795855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdd2e3-e7df-4c8f-b479-2e1d9d3f8394 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:46.841075 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:46.842166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.842641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.843102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.844354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:46.844846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:46.846528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:46.859801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 421/853] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:46.866295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.885407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:46.899093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0906c1f-4018-45a7-a944-7c0d575c65ae None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:46.913395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:46.937626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:46.938396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:46.938761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:46.939153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:46.940062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:46.940451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:46.942108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:46.955974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 433/854] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:46.956997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:46.979675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07e6612-19c4-4a3e-9258-eb772fa1986d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.017975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.019464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.019891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.020409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.021422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=a7ac2b5986a74ea9923fa0593e58a6b7) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:47.021937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.023775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.039477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 422/855] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/role_assignments?user.id=a7ac2b5986a74ea9923fa0593e58a6b7&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:47.066295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb597536-7ac2-4abb-b213-dfcdfa0b3e48 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.106419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.107240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.107565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.107849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.120474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.122055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:47.141584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:47.142507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:47.143448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:47.143448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:47.143448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:47.143448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:47.143448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 434/856] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:47.152365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.246784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:47.268373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e611a7c-5211-4ca0-a9d9-24f20cc26612 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.302785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.303475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.303762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.304044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.304765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:47.305072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.306463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc6f398-9abe-443b-9bea-4caa9ba262d0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.318853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 423/857] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:47.324922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.344650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0a1b-20c3-478e-a7b3-382b37745e96 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.344975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0a1b-20c3-478e-a7b3-382b37745e96 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.345217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0a1b-20c3-478e-a7b3-382b37745e96 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.351688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 424/858] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:47.353035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603324c-6f62-4208-b814-ccfb535bc644 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.353323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603324c-6f62-4208-b814-ccfb535bc644 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.353603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4603324c-6f62-4208-b814-ccfb535bc644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.358601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:47.373282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4603324c-6f62-4208-b814-ccfb535bc644 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:47.374481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4603324c-6f62-4208-b814-ccfb535bc644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:47.436340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b7f8bb-a2f6-4433-ba42-cd2f4fadcd1e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.451400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 425/859] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:47.453290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c67187-bd5c-4834-a8d6-9212dad66463 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.453584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c67187-bd5c-4834-a8d6-9212dad66463 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.453866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c67187-bd5c-4834-a8d6-9212dad66463 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.455323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 426/860] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:47.460831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.461357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.462358 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.481547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:47.483060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:47.483195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.483975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.484260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.484540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.492570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.494544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:47.509452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:47.510405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 435/861] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:49:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:47.511814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.557559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:47.563348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 427/862] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:47.564080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.613197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:47.629149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0ad86-fef2-470b-a1ca-1b034b1a029a None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.662984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.663667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.663951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.664225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.664981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.666319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.675030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28131476-a7df-4f67-bcd3-03de09e9a295 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.679787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 436/863] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:49:47 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:47.681326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.738945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.740338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.740642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.740925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.742119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:47.742448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.744516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.777598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 428/864] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Jun 20 05:49:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:47.779279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.813661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aef6db-77ba-48a0-bd7d-36f492fad7a8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.850445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:47.863918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.864973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.865266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.865550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:47.866219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:47.869115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:47.883432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:47.884915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:47.885772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 437/865] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Jun 20 05:49:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:47.890927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:47.905496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:47.912051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042f29a4-3ed5-4880-a32d-27e00834e78a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:47.950689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:47.951395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:47.951699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:47.951961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.016900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9ba86-1dd7-4141-805e-7fe48e9bfab4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:48.032570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.034203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.051937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:48.052623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:48.052910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:48.053186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.053891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:48.054200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.055645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.060799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 438/866] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:48.066056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:48.102521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:48.163337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 429/867] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:47 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:49:48.166767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:48.180353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:48.210554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97589e16-c0b1-4a94-b45a-669518c47268 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:48.253396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:48.254450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:48.254960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:48.255468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.256520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:48.257026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.259362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.265211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 439/868] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:48 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:48.286561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771eb142-3430-490e-9203-ea83d0724035 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:48.325235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:48.326712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:48.327274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:48.327812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.339758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.341570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:48.362616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:48.363620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:48.364492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:48.364492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:48.364492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:48.364492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:48.366838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 430/869] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:48.373479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:48.479344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74b24e2-89d8-4808-939c-57273387efc2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:48.513629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:48.514694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:48.515177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:48.515711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.516620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:48.517161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.518716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.525974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:48.537074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:48.540872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 440/870] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:48.547491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:48.644864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b452655d-e3b7-421b-bd88-3fb48116c835 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:48.679288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:48.680092 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:48.680376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:48.681222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.688005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.689481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:48.703562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:48.704442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:48.705370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 431/871] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:48.710609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:48.814405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a85b92-d1d4-42ee-a236-5c80617f27b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:48.848936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:48.849654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:48.849935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:48.850260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:48.851049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:48.851384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:48.853348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0414fab0-f2fc-4a72-b779-fa4eb8198079 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:48.865470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 441/872] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:48.871738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:48.944193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:48.980892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f922a60-1400-4707-a339-7f68d37de8bb None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:49.017948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:49.018892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.019359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.019787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.020792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=306d74f4500845a097e8e062af12dc0f) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:49.021268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:49.022854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af79d5-a7e2-4ad9-bcc8-55d65b8839ef None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:49.029525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 432/873] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:48 2022] GET /identity/v3/role_assignments?user.id=306d74f4500845a097e8e062af12dc0f&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:49.168124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54861-bd51-4218-b577-7dc52583267b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.168439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54861-bd51-4218-b577-7dc52583267b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.168722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b54861-bd51-4218-b577-7dc52583267b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.170496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 442/874] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:49.176020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.176483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.176683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.197266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:49.198459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:49.267378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 433/875] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:49.288978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b54861-bd51-4218-b577-7dc52583267b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:49.351660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:49.397082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b54861-bd51-4218-b577-7dc52583267b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:49.432309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:49.433039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.433323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.433602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.434276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:49.435666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:49.448632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:49.449661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:49.450495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:49.454837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 443/876] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:49.456259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:49.522660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:49.565340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9abb79e-01c9-440f-b447-8631fc5ac765 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:49.602227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:49.603309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.603806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.604311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.605255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:49.605788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:49.607429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde79e9f-cc25-4681-ada7-f4adb8e882d5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:49.613076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 434/877] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:49:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:49.623363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:49.729341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294ac307-a7a2-46aa-ae23-15b57b2f02f2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:49.753118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:49.766279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:49.767207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.767727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.768227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.769163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:49.769771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:49.771410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c9257-a56a-45be-92fa-1ba3cbde4f4c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:49.777035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 444/878] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:49:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:49.850472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d59c6-2b8b-406e-b609-acc49cdda4c6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.850816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d59c6-2b8b-406e-b609-acc49cdda4c6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.851041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d59c6-2b8b-406e-b609-acc49cdda4c6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.853340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 435/879] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:49.858930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:49.859227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:49.859558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:49.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:49.878522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:49.952165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 445/880] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:49.975853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d59c6-2b8b-406e-b609-acc49cdda4c6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:50.081767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d59c6-2b8b-406e-b609-acc49cdda4c6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:50.107008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:50.117653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:50.118784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.119082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.119375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.120108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:50.121495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:50.134661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:50.135706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:50.136540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:50.139729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 436/881] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:50.142228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:50.242567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445c8a9-e77c-433d-86a9-112e6a6f57e0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:50.259902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:50.280064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:50.280733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.281019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.281296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.282014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:50.282325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:50.283791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:50.293131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 446/882] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:50.294540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:50.400905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09553c-b2f5-49e7-961f-71d66644a2e6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:50.435532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:50.436207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.436542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.436824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.437537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:50.437846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:50.439278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d042ff-9931-4561-9aa2-63c117534b99 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:50.449627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 437/883] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:50.451000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:50.456797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8915be1-d8ff-4f27-9d3a-981bee3f57ad None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.457088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8915be1-d8ff-4f27-9d3a-981bee3f57ad None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.457370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8915be1-d8ff-4f27-9d3a-981bee3f57ad None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.458953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 438/884] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:50.464212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.464504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.464785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.482291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:50.483618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:50.533795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:50.563548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 439/885] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:50.570083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fbfd76-3acc-4fc3-8000-da8e40dd8f6e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:50.582078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:50.609848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:50.610774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.611063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.611441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.612696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] RBAC: Adding query filter params (interface=public, service_id=0c1fdfd1c682475e8f29aec91e29cef4, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:50.613027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:50.614822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:50.626689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 447/886] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:49:50 2022] GET /identity/v3/endpoints?service_id=0c1fdfd1c682475e8f29aec91e29cef4&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:50.627965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:50.683912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:50.691716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27cffff-1d11-4e85-a306-e25b166b6240 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:50.734315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6776814-af0f-4311-a3d1-375516a0386f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:50.735377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776814-af0f-4311-a3d1-375516a0386f None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.735757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776814-af0f-4311-a3d1-375516a0386f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.736106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776814-af0f-4311-a3d1-375516a0386f None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.737343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776814-af0f-4311-a3d1-375516a0386f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:50.739058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6776814-af0f-4311-a3d1-375516a0386f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:50.746266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc3f011-a70c-48fc-8144-cbb3a28cbe54 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:50.758649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 440/887] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:49:50 2022] POST /identity/v3/services => generated 269 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:49:50.794883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b90af7-8a86-4c98-80f3-8d66126e7551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:50.795903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90af7-8a86-4c98-80f3-8d66126e7551 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:50.796339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90af7-8a86-4c98-80f3-8d66126e7551 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:50.796765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90af7-8a86-4c98-80f3-8d66126e7551 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:50.797608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90af7-8a86-4c98-80f3-8d66126e7551 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:50.799298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90af7-8a86-4c98-80f3-8d66126e7551 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:50.811396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 448/888] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:49:50 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 05:49:51.328291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c5124-6c56-4f12-8d43-cf72ae562550 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.329330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c5124-6c56-4f12-8d43-cf72ae562550 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.329829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435c5124-6c56-4f12-8d43-cf72ae562550 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.331695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 441/889] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:51.337272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.338085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.338589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.355839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:51.357366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:51.424768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 449/890] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:51.449317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c5124-6c56-4f12-8d43-cf72ae562550 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:51.496815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:51.558964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435c5124-6c56-4f12-8d43-cf72ae562550 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:51.595322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:51.596633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.597128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.597609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.609861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:51.618406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:51.619378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:49:51.620242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:51.622891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 442/891] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Jun 20 05:49:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:51.628962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:51.662977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:51.728940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5f1a95-20f2-47e6-b196-ef95ebeb866c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:51.763120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:51.764203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.764696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.765198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.766144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:51.766679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:51.768278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67120245-bba4-4caf-9925-b44fdba7b85c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:51.776777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 450/892] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:49:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:51.782598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:51.846019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070531-004b-4599-ad7d-8a7cee2520aa None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.846312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070531-004b-4599-ad7d-8a7cee2520aa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.846519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25070531-004b-4599-ad7d-8a7cee2520aa None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.848157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 451/893] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:51.854650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.855670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.856474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.879648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:51.881873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:51.900601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172ff12-1578-4e75-8e06-73209a36cbb4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:51.902735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:51.938236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:51.939458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:51.939960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:51.940467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:51.951614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:51.953279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:51.971992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:51.973111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:51.974072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:51.974072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:51.974072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:51.974072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:51.975589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 452/894] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:51.975839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 443/895] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:51.981995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:51.997888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:52.017634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:52.109082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901aa031-8e5a-43e3-8976-6111b6978f53 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:52.115234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13cbfbf-42c8-464e-8dda-ae282dde8b39 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:52.144610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:52.145324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.145611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.145888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.146620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:52.148016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:52.161974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:52.162935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.163468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.163979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.164953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:52.165471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:52.167083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:52.169468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 444/896] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:49:51 2022] POST /identity/v3/services => generated 241 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:49:52.181894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 453/897] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:52.187955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:52.206497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:52.308536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a30116f-eaa2-4ef6-b44e-61d281d1887a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:52.346770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:52.348225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.348515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.348794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.357607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:52.359408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:52.375353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:52.376330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:52.377285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:52.378775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 445/898] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:49:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:52.384782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:52.482430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:52.497802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812c415-684b-465b-8763-9f0676111d9c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:52.531998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:52.532675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.532964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.533240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.534074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:52.534387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:52.535857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:52.554565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 454/899] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Jun 20 05:49:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:52.559852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:52.641757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:52.667852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96286b33-9b2b-4e06-a9b5-fd12e7bc7ca6 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:52.703100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87eb251-c641-49b6-b5e5-e0c0a0b858f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:52.703786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eb251-c641-49b6-b5e5-e0c0a0b858f7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.704068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eb251-c641-49b6-b5e5-e0c0a0b858f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.704345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87eb251-c641-49b6-b5e5-e0c0a0b858f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.773834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eb251-c641-49b6-b5e5-e0c0a0b858f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:52.775663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87eb251-c641-49b6-b5e5-e0c0a0b858f7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:52.846443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:52.879664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 446/900] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:49:52 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:49:52.885427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:52.925766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2aca-f7f7-477c-bd9e-a29fa575cd07 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.926079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2aca-f7f7-477c-bd9e-a29fa575cd07 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.926318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dd2aca-f7f7-477c-bd9e-a29fa575cd07 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.927968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 447/901] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:52.934148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8b589-74e2-46a0-b85f-1e30346f8ed3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:52.934435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8b589-74e2-46a0-b85f-1e30346f8ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:52.934712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8b589-74e2-46a0-b85f-1e30346f8ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:52.957673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b2a8b589-74e2-46a0-b85f-1e30346f8ed3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:52.958870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b2a8b589-74e2-46a0-b85f-1e30346f8ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:52.996745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea97f652-15f1-43f8-9f1d-ebba2940c537 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.032943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 448/902] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:53.033272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.034071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.034361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.034641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.044303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcce3e-8475-4b72-b0c7-d975096ac515 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.045010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcce3e-8475-4b72-b0c7-d975096ac515 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.045632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcce3e-8475-4b72-b0c7-d975096ac515 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.046146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.051390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.052110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 449/903] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:53 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:53.052933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.054921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.056074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.056561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:53.069941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:53.071013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:53.072297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:53.072297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:53.072297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:53.072297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.072297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 455/904] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:53.076346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.076575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:53.077921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:53.150680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 450/905] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:53.151532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.182946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.186181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667e88e1-7758-492f-bc33-410bf09048bb None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.226480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.228486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.228769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.229043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.229687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.231055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:53.242594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:53.244276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.245643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 456/906] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:53.246436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.261553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fed0d32-5f59-4db3-8e95-12d2d7c01759 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.298435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.299479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.299988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.300373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.301204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:53.301631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.303098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.305910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.322243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 451/907] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:53.322243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.347636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc4265-171c-4b12-9fd8-5af729c50f5f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.386235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.387081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.387414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.387718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.388398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.389837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:53.403499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:53.404517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.405342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 457/908] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:53.406163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.420355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.431903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4a75-6c3a-4459-bef9-c154a76a7232 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.468707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.469565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.469849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.470130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.470846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:53.471159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.472545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.482791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 452/909] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:53.482791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.519865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b75bf0-a99c-4137-b2fd-eee09da3a052 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.545048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.558163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.558961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.559270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.559555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.566253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.567783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:53.582425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:53.583347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:53.584308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 458/910] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:53.585484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.593388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6565b-b567-4f2c-b701-9a75eeb23f4f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.629946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.630617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.630912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.631193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.631949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:53.632263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.633646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.642922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 453/911] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:53.642922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.660606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.696580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd721d2-f5ce-4273-99e3-ef171c68b1a2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.732575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.733488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.733780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.734054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.734751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:53.735165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.736686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.741611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826db15-2db3-4627-9d53-243126bdceb1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.746387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 459/912] 10.222.0.45 () {56 vars in 1114 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:53.747241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.775289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.775958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.776248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.776529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.777329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:53.777638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.779071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.792270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 454/913] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:53.793063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.801960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.862054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24a308-e2e7-4a86-9f00-1db7841b5b4c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.878631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:53.898842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.900153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.900592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.900827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.901536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:53.901849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.903227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.903359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053d949-b777-42eb-9e0a-86dfd5eb07ae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:53.909181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 460/914] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:53.909807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:53.947702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:53.948368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:53.948652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:53.948931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:53.949670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:53.951031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:53.971974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 455/915] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:49:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:53.981168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:54.020063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0958131d-3077-4a1c-9785-fe871a19199f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:54.035358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:54.066904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:54.067607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.067982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.068262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.069190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] RBAC: Adding query filter params (role.id=d1f46cd037874087a9a3b99bd7924499, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=306d74f4500845a097e8e062af12dc0f) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:54.069509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:54.071755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:54.087281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 461/916] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/role_assignments?user.id=306d74f4500845a097e8e062af12dc0f&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=d1f46cd037874087a9a3b99bd7924499 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:54.130647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33307334-1630-4dd5-904f-137f08c6ab16 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:54.171859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:54.172555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.172838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.173113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.173923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] RBAC: Adding query filter params (interface=public, service_id=2b141e9ca8a640cabc1a13e3aff01a46, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:54.174244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:54.176073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bec8db-b0b9-4c3b-8e98-a50581533724 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:54.179351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:54.183890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 456/917] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:49:53 2022] GET /identity/v3/endpoints?service_id=2b141e9ca8a640cabc1a13e3aff01a46&interface=public®ion_id=RegionOne => generated 194 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:54.189307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:54.303620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33b50ee-10d7-4296-a09f-23acf1e7bb84 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:54.328988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da91d-5b9e-4fa4-9bda-0d40c2629a15 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.329296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da91d-5b9e-4fa4-9bda-0d40c2629a15 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.329558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4da91d-5b9e-4fa4-9bda-0d40c2629a15 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.331062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 457/918] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:54.337428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.337428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.337428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.347915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:54.348587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.348872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.349150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.349834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:54.351749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:54.357242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:54.358671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:54.359387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 462/919] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:49:54 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:49:54.437035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 458/920] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:54.461246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:54.507689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:54.581818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04050c7a-3bb3-4085-a282-03fc8c593759 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:54.617753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:54.618564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.618846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.619122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.619861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:54.621301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:54.634109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:54.635120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:54.636004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 463/921] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:54.643331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:54.744003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680f1ef-72d2-477b-b842-72c1c93dc856 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:54.771935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:54.780911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:54.781583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.781870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.782147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.782862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:54.783172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:54.784557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:54.789677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 459/922] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:49:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:54.795397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:54.914427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:54.918002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0209ae-364f-4748-9117-a407be019742 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:54.953614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:54.954611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:54.955105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:54.955639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:54.956547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:54.957061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:54.958659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:54.964267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 464/923] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:49:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:54.971305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:55.075541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb02c7-0ec8-4950-80c9-7616ab523889 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:55.109988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:55.110655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:55.110943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:55.111222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:55.112035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] RBAC: Adding query filter params (interface=public, service_id=bafee45c1ffc450c90d360e95ba9e607, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:55.112352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:55.113713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0516fe57-0d8c-4fa9-99a8-07751d36261c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:55.119760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 460/924] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:49:54 2022] GET /identity/v3/endpoints?service_id=bafee45c1ffc450c90d360e95ba9e607&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:55.125254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:55.235032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6253f0eb-b43a-421b-beee-e240c26c6e86 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:55.271031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ebf2a8-b1ad-4e65-9c9c-30945b53f627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:55.272144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf2a8-b1ad-4e65-9c9c-30945b53f627 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:55.272669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf2a8-b1ad-4e65-9c9c-30945b53f627 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:55.273181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ebf2a8-b1ad-4e65-9c9c-30945b53f627 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:55.274598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf2a8-b1ad-4e65-9c9c-30945b53f627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:55.276197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ebf2a8-b1ad-4e65-9c9c-30945b53f627 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:55.282854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 465/925] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:49:55 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:49:56.167156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aac58-ff71-4a5c-8a1f-c167429c7f36 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.167729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aac58-ff71-4a5c-8a1f-c167429c7f36 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.167729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1aac58-ff71-4a5c-8a1f-c167429c7f36 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.169377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 461/926] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:56.174627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.175050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.175233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.181783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b868-21e3-445e-ba49-fb1c5be48ba8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.182078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b868-21e3-445e-ba49-fb1c5be48ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.182365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e552b868-21e3-445e-ba49-fb1c5be48ba8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.184112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 462/927] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:56.191793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.192087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.192369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.194370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:56.195915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:56.209719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:56.210910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:56.225291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:56.283274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 466/928] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:56.300034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.302390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:56.309225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 463/929] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:56.332990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.410685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739893a-bfd0-484f-8a70-985c25ed59cd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.444977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c847b74-b3c0-4f72-9664-16e2b1651a9a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.446823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:56.447522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.447810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.448091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.448804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:56.450244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:56.451640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING keystone.server.flask.common [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 20 05:49:56.471103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 05:49:56.471103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 20 05:49:56.471103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [parameters: {'id': '432bd70ecab1422091228137ecfb4c55', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 20 05:49:56.471103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 05:49:56.486125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3652ace0-5fa4-463d-b91f-d20b8e6b7236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:56.487211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ace0-5fa4-463d-b91f-d20b8e6b7236 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.487675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ace0-5fa4-463d-b91f-d20b8e6b7236 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-66ab2907-e303-4a11-b886-10c286a08f69 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 432bd70ecab1422091228137ecfb4c55.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 432bd70ecab1422091228137ecfb4c55. Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 05:49:56.487950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 20 05:49:56.488916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Jun 20 05:49:56.490099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 05:49:56.491206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 05:49:56.492373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 05:49:56.493393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'id': '432bd70ecab1422091228137ecfb4c55', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.494452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:56.495512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 432bd70ecab1422091228137ecfb4c55. Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 467/930] 10.222.0.45 () {60 vars in 1140 bytes} [Mon Jun 20 05:49:56 2022] POST /identity/v3/projects => generated 262 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f7043-b713-47bd-8d4b-7cdc3fa5d88f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.496673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f7043-b713-47bd-8d4b-7cdc3fa5d88f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.498237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95f7043-b713-47bd-8d4b-7cdc3fa5d88f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.498321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652ace0-5fa4-463d-b91f-d20b8e6b7236 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.502157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 468/931] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:56.502889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f7043-b713-47bd-8d4b-7cdc3fa5d88f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.512446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3652ace0-5fa4-463d-b91f-d20b8e6b7236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3652ace0-5fa4-463d-b91f-d20b8e6b7236 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:49:56.521081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:56.522044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:49:56.522936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:56.525020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:56.527548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 464/932] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:56.528855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.529147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.529429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.547040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:56.548253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:56.593474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:56.615855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95f7043-b713-47bd-8d4b-7cdc3fa5d88f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.619855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 465/933] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:56.624071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.652273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:56.653068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.653358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.653637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.660306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:56.661872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:56.675936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:56.677027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.677956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 469/934] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:56.678439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.740997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce22a77-82c7-48fa-a365-3e8938d738f0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.761590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:56.784085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:56.785043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.785335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.785617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.786615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:56.786924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:56.788047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85554829-f180-4b79-afef-a33b722633d0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.788392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:56.803081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 466/935] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:56.803081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.825610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:56.826621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.826907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.827186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.827881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:56.829262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:56.842454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:56.843463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:56.844593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 470/936] 10.222.0.45 () {56 vars in 1132 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:56.845466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:56.858935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:56.921813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379f351-87f9-4bb2-9413-c5a498165f71 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.955322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:56.956096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:56.956449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:56.956795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:56.957623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:56.958001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:56.959467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a126020f-ee81-4a41-98d8-462067d773e9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:56.959594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:56.987285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 467/937] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:56.987285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.008644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.009440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.009728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.010006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.019444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.025425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.027203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:57.049476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:57.050632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:57.051704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:57.051704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:57.051704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:57.051704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:57.053292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 471/938] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:57.060043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.109449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.114013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93249444-6dcb-49a3-8d0c-d7ad97dcc1e4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.154545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.155672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.156214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.156723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.157656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:57.158178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.160023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.165948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 468/939] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:57.171671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.199530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44370a59-b4bf-4d30-9f43-5865577e33ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.238171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.238863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.239196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.239556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.240323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:57.240660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.242050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.257616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 472/940] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:57.263873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.293147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3570093-22d2-4f56-9be0-6fd29c32c27b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.329184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.338250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.339454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.339928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.340490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.341847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:57.342418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.344259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15be24-8a74-46b5-b20a-142709ed0489 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.349684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 469/941] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:57.351083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36407a-a64d-4ecd-82e0-0bd266bff599 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.351411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36407a-a64d-4ecd-82e0-0bd266bff599 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.351701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c36407a-a64d-4ecd-82e0-0bd266bff599 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.353183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 470/942] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:57.356603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36407a-a64d-4ecd-82e0-0bd266bff599 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.386877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.388462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3e503f-6dfd-49fd-873d-99e649061121 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.425064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.425900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.426369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.426605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.435199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.436708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f6ff30a1-3ed6-4daa-8e68-15ab72f6bd44 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:57.452706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:57.453881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:57.454851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 473/943] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:57.455903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.455903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.455903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.472913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:57.474236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:57.476781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c36407a-a64d-4ecd-82e0-0bd266bff599 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.513453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.514169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.514405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.514684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.515459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.516824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.539054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 471/944] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:49:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:57.544301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.571754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 474/945] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:57.586294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.629854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.662402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.686422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8321a6a5-ee90-4475-82ea-ef19782ffdec None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.710856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616d6d6-bde0-4757-a613-6f691af03d48 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.731139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.732343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.732472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.732991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.733897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:57.734207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.735989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.760714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 472/946] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:57.761616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.772878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.774181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.774472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.774754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.775875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.777991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:57.796114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:57.797244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:57.798145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 475/947] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:57.806621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.882805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1cb71-e847-4edb-83ba-e81a9a8d250f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.918119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.921664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02dbe3c-4d7d-48c7-9ff9-a1ed3d1589a3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:57.921956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.922807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.923235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.923612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.933921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:57.936613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.938432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:49:57.957900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:57.959066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:57.960212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:49:57.960212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:49:57.960212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:49:57.960212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:57.963328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 473/948] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:57.965425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:57.967113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:57.970236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:57.970674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:57.971125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:57.972111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:57.972684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:57.974283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:57.980627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 476/949] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:57.987356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:58.084652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ecde58-6580-4d17-8be2-78fa004816e9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:58.089459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe58cc-72b7-4a6e-8dd3-73988afcae1d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:58.123779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:58.124542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:58.124831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:58.125121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:58.125555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:58.125852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:58.126166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:58.126899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:58.127452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:58.127827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:58.128174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:58.129083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:58.129612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:58.131176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:58.141325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 477/950] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:58.142445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:58.143346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 474/951] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:49:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:49:58.148075 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:58.181957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:58.222008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:58.275138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e52bec-be7d-46db-9fc1-54133cadea03 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:58.280035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcc290-2888-4f42-a5ee-3ac6eb31c792 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:58.313472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:58.314316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:58.314698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:58.315075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:58.315592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:58.315861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:58.316390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:58.316677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:58.316954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:58.317313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:58.324886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:58.326521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:58.339166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 478/952] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:49:58 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:49:58.342164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:58.343218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:58.344666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 475/953] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:58.349648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:58.448630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:58.476842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050727e5-5937-4d75-8c00-c8d1cd0c8491 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:58.513776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:58.514445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:58.514734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:58.515017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:58.515841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:58.516164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:58.517675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9336da9d-4bcf-41ff-9d3a-17f0c31df164 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:58.530933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 479/954] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:49:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:49:58.536656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:58.641182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a46380-fa9a-4ba0-94f8-11353002103f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:58.655167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:58.676594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3aa51-9a33-4c6f-a0d3-eca5f93997ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:58.677492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3aa51-9a33-4c6f-a0d3-eca5f93997ae None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:58.677775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3aa51-9a33-4c6f-a0d3-eca5f93997ae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:58.678052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3aa51-9a33-4c6f-a0d3-eca5f93997ae None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/306d74f4500845a097e8e062af12dc0f/roles/d1f46cd037874087a9a3b99bd7924499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:58.680945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3aa51-9a33-4c6f-a0d3-eca5f93997ae None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=306d74f4500845a097e8e062af12dc0f, role_id=d1f46cd037874087a9a3b99bd7924499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:58.682374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc3aa51-9a33-4c6f-a0d3-eca5f93997ae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:58.698461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 476/955] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:49:58 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/306d74f4500845a097e8e062af12dc0f/roles/d1f46cd037874087a9a3b99bd7924499 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:49:59.265897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f177c94-e8b8-4870-9c81-9526973dc652 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.266627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f177c94-e8b8-4870-9c81-9526973dc652 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.267136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f177c94-e8b8-4870-9c81-9526973dc652 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.269084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 480/956] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:49:59.275141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.276054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.276574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.297208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:59.298715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:59.366560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 477/957] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:49:59.391830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f177c94-e8b8-4870-9c81-9526973dc652 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:59.502173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f177c94-e8b8-4870-9c81-9526973dc652 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:59.541068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:59.542076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.542590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.543087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.544081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:59.545736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:59.561083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.sql.core [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 05:49:59.561083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 20 05:49:59.561083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [parameters: {'user_id': '4b11759d6b5146d8b070942200c30d6e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 20 05:49:59.561083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80160) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-71ce2a7a-cd99-4e01-be78-4c2f768c22e1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jun 20 05:49:59.569706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 05:49:59.571195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application yield resource Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application next(self.gen) Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.rollback() Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.force_reraise() Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise self.value Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application session.commit() Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 20 05:49:59.572577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._prepare_impl() Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.session.flush() Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._flush(objects) Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application compat.raise_( Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application flush_context.execute() Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rec.execute(self) Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 20 05:49:59.573851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 20 05:49:59.574855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result = self._query(query) Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application conn.query(q) Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application result.read() Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 20 05:49:59.576464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [parameters: {'user_id': '4b11759d6b5146d8b070942200c30d6e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:59.578575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 20 05:49:59.580825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.584799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 481/958] 10.222.0.45 () {60 vars in 1131 bytes} [Mon Jun 20 05:49:59 2022] POST /identity/v3/users => generated 160 bytes in 192 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:59.596137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:59.652580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339daca-8d19-4ba4-a1ae-6720009b28c2 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.652914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339daca-8d19-4ba4-a1ae-6720009b28c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.653145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339daca-8d19-4ba4-a1ae-6720009b28c2 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.657647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 482/959] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:49:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:49:59.662475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.662769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.663050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.675279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:59.686595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:49:59.688198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:49:59.697611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e473995-807e-40b1-8aa0-67153a59c1a5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:59.728526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:59.733004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:59.733819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.735301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.735627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.744740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:59.746177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:49:59.761891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:59.762870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:49:59.763886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 478/961] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:49:59 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:49:59.764525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 483/961] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:49:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:49:59.770155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:59.781184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:59.881823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396d73f3-8e8a-419c-a504-87d4f5bccb44 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:59.900561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac44a3f9-6ed9-40a6-8223-4ea3a6022d05 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:49:59.918167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:59.919386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.919885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.920408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.921433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:49:59.921989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:59.923719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:59.939620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:49:59.945461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:49:59.946283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:49:59.946569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:49:59.946855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:49:59.947548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:49:59.949121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:49:59.961385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:49:59.962447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:49:59.963375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:49:59.967951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 484/962] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:49:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:49:59.969174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:49:59.971225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 479/963] 10.222.0.45 () {56 vars in 1101 bytes} [Mon Jun 20 05:49:59 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:00.092852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e66fcc-da1a-4a5d-9281-65ab09f2a36f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:00.135170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:00.135867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.136153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.136434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.137141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:00.137453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:00.138825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf16bfb3-df83-4751-ad3a-41d0733ba44f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:00.144270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 485/964] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:49:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:00.150102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:00.174057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:00.260602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921ad46-995c-400d-8018-a06deaa04649 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:00.294886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:00.295707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.295888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.296146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.296857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:00.297169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:00.298534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bb51ec-2864-4747-8de6-e8595ad82987 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:00.311348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 480/965] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:50:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:00.531260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4d696-fddd-4894-b640-3614026ffe5c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.531775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4d696-fddd-4894-b640-3614026ffe5c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.531877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4d696-fddd-4894-b640-3614026ffe5c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.533341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 486/966] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:00.538330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.538633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.538922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.556333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:00.557523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:00.626798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 481/967] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:00.649332 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4d696-fddd-4894-b640-3614026ffe5c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:00.709876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:00.755177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4d696-fddd-4894-b640-3614026ffe5c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:00.788040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:00.788775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.789059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.789864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.790516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:00.791926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a852e35f-b7fe-451e-8af2-9ead399b01d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:00.803926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:00.805518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:50:00.806360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:00.810684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 487/968] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:50:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:00.811554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:00.822443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85893e-0813-46ca-9bca-866f02d0a9dc None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.822443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85893e-0813-46ca-9bca-866f02d0a9dc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.822443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85893e-0813-46ca-9bca-866f02d0a9dc None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.823209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 488/969] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:50:00.827919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.828218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.828505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.835714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:00.846090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:00.847307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:00.918028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 489/970] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:00.921979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1312d14-eee1-4952-88a9-44dd51a307b1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:00.941771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:00.956748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:00.957416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:00.957698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:00.957976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:00.958681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:00.958994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:00.960389 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:00.970013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 482/971] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:50:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:00.971021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.004503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.050600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3099d33-abbb-4ef6-a1eb-7c3dbb652daa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.051694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.074944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630955ec-6f29-49d4-9ba1-4226280188ab None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.086925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.087867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.088158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.088435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.103221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:01.109466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.110129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.110425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.110702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.111441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:01.111757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:01.112382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:01.113292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:01.114138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:01.114807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.114885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 490/972] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Jun 20 05:50:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:01.119338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.123310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 483/973] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:50:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:01.132310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.222920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa294dec-01ec-453e-9a6f-d5b6bb0a78d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.235657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88c46cc-9fdf-439f-8f2f-8bd5ab901325 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.237079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.258845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.259593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.259880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.260210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.260924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:01.261231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.262654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.275651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 491/974] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:01.276635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.277817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.278483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.278783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.279055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.279890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] RBAC: Adding query filter params (interface=admin, service_id=bafee45c1ffc450c90d360e95ba9e607, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:01.280212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.281572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.291124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 484/975] 10.222.0.45 () {56 vars in 1249 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/endpoints?service_id=bafee45c1ffc450c90d360e95ba9e607&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:01.292339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.300499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.388945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66dbc4-075a-4a15-b980-f9c215c67d85 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.399741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac563b5-3f3f-438c-993b-81b7d46e8e43 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.428064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.428861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.429154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.429438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.435431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.436335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.437188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.437667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.438937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.440169 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.441009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.441603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.448201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 485/976] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:01.459845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:01.460824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:01.461851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:01.461851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:01.461851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:01.461851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:01.463610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 492/977] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:01.467046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.543775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.577940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63857a18-e5db-4922-8947-7132f5837691 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.615639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.616316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.616604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.616889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.617613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:01.617926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.619306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.631113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 486/978] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:01.637422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.666856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.741248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c12aaa3-5ba0-48a2-a7f2-a737b55a5851 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.775795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.776617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.776905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.777199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.784904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.786346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:50:01.801448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:01.802418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:01.803427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 493/979] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:01.809772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:01.918621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4ac6cd-a00b-4d31-8b57-c0cce0f3a9a3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:01.946638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:01.953679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:01.954360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:01.954647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:01.954926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:01.955800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:01.956114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:01.957530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:01.980568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 487/980] 10.222.0.45 () {56 vars in 1149 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:01.981782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:02.068286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:02.082470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad593b3-070c-4797-b004-5230260ff438 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:02.116185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c63ca3f-9b4e-411c-9748-5e9ab8e7c05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:02.116836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ca3f-9b4e-411c-9748-5e9ab8e7c05b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.117122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ca3f-9b4e-411c-9748-5e9ab8e7c05b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.117428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63ca3f-9b4e-411c-9748-5e9ab8e7c05b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.184336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63ca3f-9b4e-411c-9748-5e9ab8e7c05b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:02.185714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c63ca3f-9b4e-411c-9748-5e9ab8e7c05b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:02.270689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:02.290469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 494/981] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:01 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 05:50:02.291752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:02.322303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4732b7-8baf-4b0a-b066-57f71f1a8d27 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.322602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4732b7-8baf-4b0a-b066-57f71f1a8d27 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.323087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4732b7-8baf-4b0a-b066-57f71f1a8d27 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.324586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 495/982] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:50:02.329223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.329520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.329805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.346693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:02.347922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:02.393726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf6d4f-a735-42cd-9c29-551e991531e3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:02.422552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 496/983] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:02.430599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:02.431422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.431711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.431992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.442242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:02.443700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:02.444564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:02.463026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:02.464126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:02.465044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:02.465044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:02.465044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:02.465044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:02.465044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 488/984] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:02.469873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:02.513158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:02.553256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652fda43-33e0-4264-90a9-feb5f619c545 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:02.572210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:02.580296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137db96f-1b5f-42da-a23b-2d5622ce0476 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:02.591054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:02.591747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.592023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.592343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.593024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:02.594661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:02.614559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 497/985] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:50:02 2022] POST /identity/v3/services => generated 242 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:50:02.635403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:02.636091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.636374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.636648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.637383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:02.637687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:02.639104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:02.653037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 489/986] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:02.658724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:02.771978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad43e6c-0147-4f2a-a93b-5cb8bfd7ddb9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:02.808068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:02.808868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.809153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.809433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.821906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:02.823734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:50:02.838314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:02.839209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:02.840908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 498/987] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:50:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:02.845715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:02.945202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:02.951533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1ebc2c-7f63-4308-bbb9-ef2fb28b3be8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:02.985983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:02.986651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:02.986933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:02.987208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:02.988020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:02.988377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:02.989848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e8bf86-d591-431f-92f6-ffc8729deb9a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:03.007295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 490/988] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:50:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:03.008531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:03.049613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:03.112627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499c9ac0-a3c4-4bc4-a8a2-b04947c770e7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:03.147004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:03.147691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:03.147972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:03.148246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:03.149595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] RBAC: Adding query filter params (role.id=d1f46cd037874087a9a3b99bd7924499, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=306d74f4500845a097e8e062af12dc0f) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:03.149906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:03.151358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab3d02-1379-4a1f-8b50-9014d84bc5dc None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:03.163293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 499/989] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:03 2022] GET /identity/v3/role_assignments?user.id=306d74f4500845a097e8e062af12dc0f&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=d1f46cd037874087a9a3b99bd7924499 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:03.495800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bc934-e1f2-4951-9aed-db40feb12e6e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:03.496324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bc934-e1f2-4951-9aed-db40feb12e6e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:03.496402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9bc934-e1f2-4951-9aed-db40feb12e6e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:03.497859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 491/990] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:03.502842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:03.503139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:03.503446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:03.520054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:03.521244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:03.595728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 500/991] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:03.613847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bc934-e1f2-4951-9aed-db40feb12e6e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:03.720458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9bc934-e1f2-4951-9aed-db40feb12e6e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:03.757188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:03.758835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:03.759429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:03.759934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:03.760837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:03.762394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:03.775312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:03.776301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 20 05:50:03.777181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:03.782612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 492/992] 10.222.0.45 () {56 vars in 1126 bytes} [Mon Jun 20 05:50:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:03.783795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:03.821402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:03.895355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ed2cf-3d66-4316-8ca2-4e21428f595c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:03.929040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:03.930233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:03.930520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:03.930799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:03.931537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:03.931852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:03.933225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:03.942636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 501/993] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:50:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:03.943887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:03.967434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:04.047172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4f4b90-6d20-4346-bb49-449fe33d221c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:04.080554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:04.081219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:04.081508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:04.081789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:04.082495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:04.082806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:04.084214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cdc55-7e4f-4e1a-a013-d5e321778a40 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:04.094754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 493/994] 10.222.0.45 () {56 vars in 1122 bytes} [Mon Jun 20 05:50:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:04.096036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:04.194233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:04.201508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ab57ff-823c-4731-8027-ee4a01378a68 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:04.236284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f7151-80df-40f0-81c6-38dfacb7e303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:04.236948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f7151-80df-40f0-81c6-38dfacb7e303 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:04.237237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f7151-80df-40f0-81c6-38dfacb7e303 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:04.237518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f7151-80df-40f0-81c6-38dfacb7e303 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:04.238253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f7151-80df-40f0-81c6-38dfacb7e303 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:04.239654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f7151-80df-40f0-81c6-38dfacb7e303 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:04.263286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 502/995] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:50:04 2022] POST /identity/v3/endpoints => generated 325 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:04.663272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b6ac-7753-442f-a109-1aff98b74c85 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:04.663828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b6ac-7753-442f-a109-1aff98b74c85 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:04.663828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14b6ac-7753-442f-a109-1aff98b74c85 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:04.665431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 494/996] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:04.670495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:04.670788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:04.671064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:04.688002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:04.689184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:04.762601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 503/997] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:04 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:04.784938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14b6ac-7753-442f-a109-1aff98b74c85 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:04.875060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:04.893060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14b6ac-7753-442f-a109-1aff98b74c85 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:04.930036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:04.930843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:04.931133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:04.931446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:04.932100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:04.933466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:04.945376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:04.946473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:04.947362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 495/998] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:50:04 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:04.952558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.036675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:05.057889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec29d4fe-3b7a-4afd-95b8-22465a0f5e1e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.092457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.093131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.093421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.093958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.094667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:05.094979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.096391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b0679-338b-433f-bed7-ee0d02f52326 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.105690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 504/999] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:50:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:05.107218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.207650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a97f0a-af2d-45d2-8ad8-fe4a4af504c7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.240947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459f33c-8628-45a3-812d-cd20581c57bc None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.241232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459f33c-8628-45a3-812d-cd20581c57bc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.241512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459f33c-8628-45a3-812d-cd20581c57bc None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.242923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 505/1000] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:05.244945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.245613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.245897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.246173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.246887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:05.247198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.248044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.248341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.248583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.248681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.254337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 496/1001] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:50:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:05.260546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.267221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:05.268449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:05.276609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:05.340096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 506/1002] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:05 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:05.362343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.371338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ed5774-f5e1-4a47-9176-d1b4cf2c4ad9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.411855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.412623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.412977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.413338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.414247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] RBAC: Adding query filter params (interface=public, service_id=72fd350bd74d424aaedf0859b42da2b8, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:05.414647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.416051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.421521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 497/1003] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:50:05 2022] GET /identity/v3/endpoints?service_id=72fd350bd74d424aaedf0859b42da2b8&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:05.426781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.447672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:05.472557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb39cf2-20fe-4dc8-b974-241ab2601131 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.507617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.509220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.509507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.509785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.510444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.510813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:05.512292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:05.525162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:05.526377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:05.527237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 507/1004] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Jun 20 05:50:05 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:05.532697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.540179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2270e8e-1d05-4689-8a10-2133d81ba969 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.575704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.576366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.576657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.576937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.577624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.578979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.585385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 498/1005] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:50:05 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:50:05.652349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1b114d-1811-4c48-9ee6-0b37f759bd51 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.688265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.688950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.689236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.689519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.690239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:05.690552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.691978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da50de7b-0ddb-41bf-b836-cb592776c50f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.702708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 508/1006] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:05 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:05.704237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:05.812380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc9c587-a1cf-4dd8-a0ff-3461bd07eb60 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:05.849937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:05.850612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:05.850897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:05.851174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:05.852446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] RBAC: Adding query filter params (type=image) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:05.852760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:05.854120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e0b4d-9fbd-46ec-82bf-ac78337cc029 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:05.858997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 499/1007] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:05 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:06.314804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22401-5350-464b-9653-e90b08e6904e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:06.315336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22401-5350-464b-9653-e90b08e6904e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:06.315447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c22401-5350-464b-9653-e90b08e6904e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:06.316941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 509/1008] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:06.321943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:06.322248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:06.322534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:06.328482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:06.340429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:06.341627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:06.411408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 500/1009] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:06.431604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22401-5350-464b-9653-e90b08e6904e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:06.435430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:06.535730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c22401-5350-464b-9653-e90b08e6904e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:06.569395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc8a68-0a85-4638-ad57-12be8e255c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:06.570638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8a68-0a85-4638-ad57-12be8e255c42 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:06.570925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8a68-0a85-4638-ad57-12be8e255c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:06.571204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc8a68-0a85-4638-ad57-12be8e255c42 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:06.572404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc8a68-0a85-4638-ad57-12be8e255c42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:06.573847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc8a68-0a85-4638-ad57-12be8e255c42 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:06.575531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 510/1010] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:06.581233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:06.683892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cecdb-00ae-4615-be9e-f3a2bd559af2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:06.720622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b7f5-01a9-4aa8-b44b-3158febfbfa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:06.721962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b7f5-01a9-4aa8-b44b-3158febfbfa3 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:06.722401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b7f5-01a9-4aa8-b44b-3158febfbfa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:06.722830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b7f5-01a9-4aa8-b44b-3158febfbfa3 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:06.723743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b7f5-01a9-4aa8-b44b-3158febfbfa3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:06.725339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b7f5-01a9-4aa8-b44b-3158febfbfa3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:06.728849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:06.753228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 501/1011] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:50:06 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:07.635485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d741b27-9cfd-4d76-8880-c092790c6adc None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:07.636480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d741b27-9cfd-4d76-8880-c092790c6adc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:07.637065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d741b27-9cfd-4d76-8880-c092790c6adc None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:07.638637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 511/1012] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:07.644046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779e64-c242-4bdd-a93f-0389b5d92658 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:07.644346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779e64-c242-4bdd-a93f-0389b5d92658 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:07.644629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b779e64-c242-4bdd-a93f-0389b5d92658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:07.661790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7b779e64-c242-4bdd-a93f-0389b5d92658 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:07.663001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7b779e64-c242-4bdd-a93f-0389b5d92658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:07.730804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 502/1013] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:07.754122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d741b27-9cfd-4d76-8880-c092790c6adc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:07.833968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:07.866763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d741b27-9cfd-4d76-8880-c092790c6adc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:07.903894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:07.904617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:07.904905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:07.905189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:07.905849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:07.907235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:07.909539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3fa45-ef5c-41d7-9d90-6f1329371983 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:07.909813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3fa45-ef5c-41d7-9d90-6f1329371983 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:07.910105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3fa45-ef5c-41d7-9d90-6f1329371983 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:07.911523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 503/1014] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:50:07.916933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:07.917216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:07.917499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:07.920214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:07.921230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:07.927190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 512/1015] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:50:07 2022] GET /identity/v3/services/compute => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:07.928373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:07.942061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:07.943616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:08.012392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 504/1016] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:07 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:08.031759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ff2717-eb87-4b98-84aa-13c1ed4050c2 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:08.034932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:08.067501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:08.068156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:08.068439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:08.068714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:08.069413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] RBAC: Adding query filter params (name=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:08.069719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:08.071654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:08.073420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:08.082154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:08.083324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 513/1017] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:50:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:08.084308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:08.142964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b180f91c-a1ca-4af2-b850-b394c6e1adcf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:08.178493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:08.179284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:08.179664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:08.179940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:08.180675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:08.182027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:08.189766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe68cc7-69b6-4c6b-9754-a1b8991328ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:08.196189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 505/1018] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:50:08 2022] POST /identity/v3/services => generated 242 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:50:08.240112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:08.240796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:08.241085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:08.241366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:08.242136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:08.242454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:08.244372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e915-ba96-488f-bfbe-b508d7ab95e3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:08.250394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 514/1019] 10.222.0.45 () {56 vars in 1112 bytes} [Mon Jun 20 05:50:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:08.256352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:08.368206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2396057f-e673-4ceb-8b64-6d897d373d40 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:08.375373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:08.406722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0c994-f671-4fb5-930a-c27c98dd4b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:08.407415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0c994-f671-4fb5-930a-c27c98dd4b6d None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:08.407703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0c994-f671-4fb5-930a-c27c98dd4b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:08.407976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0c994-f671-4fb5-930a-c27c98dd4b6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:08.408722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0c994-f671-4fb5-930a-c27c98dd4b6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:08.410080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0c994-f671-4fb5-930a-c27c98dd4b6d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:08.436094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 506/1020] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:50:08 2022] POST /identity/v3/endpoints => generated 334 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:50:08.834954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883283b8-9530-489d-b2c4-23285f8d6987 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:08.835511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883283b8-9530-489d-b2c4-23285f8d6987 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:08.836340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883283b8-9530-489d-b2c4-23285f8d6987 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:08.837779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 515/1021] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:08.843206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:08.843962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:08.844198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:08.861013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:08.862203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:08.938054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 507/1022] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:08.965041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883283b8-9530-489d-b2c4-23285f8d6987 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:09.029119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:09.073497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883283b8-9530-489d-b2c4-23285f8d6987 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:09.108951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b953ee2-574f-4e5f-88f3-81eb23674d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:09.109748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b953ee2-574f-4e5f-88f3-81eb23674d7f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:09.110032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b953ee2-574f-4e5f-88f3-81eb23674d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:09.110308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b953ee2-574f-4e5f-88f3-81eb23674d7f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:09.111512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b953ee2-574f-4e5f-88f3-81eb23674d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:09.112994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b953ee2-574f-4e5f-88f3-81eb23674d7f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:09.119074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 516/1023] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:09.120329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:09.222428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:09.235340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47dcc38-2c3f-449f-a06c-f3da17aadefb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:09.270858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442dbe2-6602-44e5-9ca1-83a37aad2cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:09.271567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442dbe2-6602-44e5-9ca1-83a37aad2cba None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:09.271859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442dbe2-6602-44e5-9ca1-83a37aad2cba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:09.272137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442dbe2-6602-44e5-9ca1-83a37aad2cba None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:09.272861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442dbe2-6602-44e5-9ca1-83a37aad2cba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:09.274323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442dbe2-6602-44e5-9ca1-83a37aad2cba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:09.299282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 508/1024] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:50:09 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:50:10.345145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900954-f929-4ffc-9602-8a113eeb8155 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.345666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900954-f929-4ffc-9602-8a113eeb8155 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.345744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93900954-f929-4ffc-9602-8a113eeb8155 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.347110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 517/1025] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:10.352728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1085cae-f779-49a3-8992-36c2dc766713 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.353120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1085cae-f779-49a3-8992-36c2dc766713 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.353304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1085cae-f779-49a3-8992-36c2dc766713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.370155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d1085cae-f779-49a3-8992-36c2dc766713 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:10.371379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d1085cae-f779-49a3-8992-36c2dc766713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:10.442256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 509/1026] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:10.465808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93900954-f929-4ffc-9602-8a113eeb8155 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:10.568425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93900954-f929-4ffc-9602-8a113eeb8155 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:10.576327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e480d-1b01-46ba-9abd-ef563e0d0c26 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.576625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e480d-1b01-46ba-9abd-ef563e0d0c26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.576918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0e480d-1b01-46ba-9abd-ef563e0d0c26 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.578418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 510/1027] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:50:10.583231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.583563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.583843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.597759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:10.602641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:10.603871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:10.607812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:10.608610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.608900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.609180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.609843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:10.611657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:10.623792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:10.624817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:10.628373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 518/1028] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Jun 20 05:50:10 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:10.630716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:10.679663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 511/1029] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:10.700552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:10.704049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:10.740762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b67855-c38b-4890-9cac-579fdb7a7bc3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:10.777109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:10.777785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.778074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.778354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.779061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:10.779569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:10.780998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:10.791098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 519/1030] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:10.791956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:10.801805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa9332-2971-4ae8-8a5a-1dcf8fb9eb7a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:10.843147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:10.844257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.844544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.844823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.854825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:10.859292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:10.863456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:10.864384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:10.865366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 512/1031] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Jun 20 05:50:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:10.871228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:10.910570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cc524c-7287-4b46-b27e-62f7464998e9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:10.937966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:10.948353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:10.949027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:10.949318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:10.949601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:10.950313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] RBAC: Adding query filter params (type=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:10.950629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:10.952041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:10.961986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 520/1032] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:10.962858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:10.978810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcc682-70ab-4c78-8e08-48792d70e587 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.014505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.015175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.015496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.015779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.016528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:11.016839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.018212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.026664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 513/1033] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:50:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:11.032170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.074878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb7b58-a635-4907-a5f7-00c88076091e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.096628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.112140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.112783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.113079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.113352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.114149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] RBAC: Adding query filter params (interface=public, service_id=49c1fce8eebe4f43b79118859d82e606, region_id=RegionOne) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:11.114463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.115872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.123279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 521/1034] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:50:10 2022] GET /identity/v3/endpoints?service_id=49c1fce8eebe4f43b79118859d82e606&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:11.127058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.143562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf601b4-7dbc-4edf-9738-13d92e77f8cd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.170194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.179455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.180255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.180544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.180825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.192619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.194064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:11.217515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:11.218751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:11.219686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:11.219686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:11.219686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:11.219686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:11.219686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 514/1035] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:11.224634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.232443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730917b5-84d9-4d00-a0e8-f788df6d12d1 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.269762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9431726-9e2c-4485-87dd-1302ec7e08ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.270433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9431726-9e2c-4485-87dd-1302ec7e08ed None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.270720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9431726-9e2c-4485-87dd-1302ec7e08ed None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.271001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9431726-9e2c-4485-87dd-1302ec7e08ed None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.271727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9431726-9e2c-4485-87dd-1302ec7e08ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.273652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9431726-9e2c-4485-87dd-1302ec7e08ed None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.280517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 522/1036] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 05:50:11.340873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873bd7a9-17ef-4ac4-aa6d-4d14cff5c21c None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.357008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8c55-2378-4058-815d-62b8ee9d64e5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.357540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8c55-2378-4058-815d-62b8ee9d64e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.357540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f8c55-2378-4058-815d-62b8ee9d64e5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.359094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 523/1037] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:50:11.364682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.364969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.365164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.378801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.379528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.379788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.380063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.380790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:11.381104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.382848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:11.383042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.384094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:11.396249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 515/1038] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:11.402712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.439470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.448633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.467060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 524/1039] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:11.490491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.513090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5548b92-2f47-459a-a6a0-8063023936de None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.548521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.549421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.549716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.549997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.557979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.559689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:50:11.575442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:11.576489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:11.578000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 516/1040] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:11.582442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.596951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b8f4bf-1870-4ad5-a511-713e5f633e1c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.632931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.633693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.633974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.634590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.646258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:11.653835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:11.654778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:11.655702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 525/1041] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:11.660673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.690890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.695679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840d912-eceb-4959-9a33-bd510dfdc41e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.701615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.739592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.741022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.741550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.742702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.743616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:11.743931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.745367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.767091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 517/1042] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:11.768113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.771491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e950b32-f562-4409-a4d0-825fa59d0673 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.806534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.807200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.807510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.807784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.808579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:11.808884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.810288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.828640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 526/1043] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:11.829425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:11.881536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b6d5f-8ac9-4e1d-b2c3-ddeaa28791e2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.919703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.920391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.920663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.921213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.933298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.936989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f561a5-3a9b-416b-b990-57b81561e796 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:11.947718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:11.974105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:11.974989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:11.975305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:11.975589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] PATH_INFO: `/v3/users/26d38509c5294b9e92c69fd03a39fa39` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:11.990365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] RBAC: Authorizing `identity:get_user(user_id=26d38509c5294b9e92c69fd03a39fa39)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.992492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.994522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 527/1044] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/users/26d38509c5294b9e92c69fd03a39fa39 => generated 285 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:11.995016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:11.996509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:11.999803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.100826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 518/1045] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:12.106742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.107783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e0c6c0-7976-410c-a3cf-a2303c5c0e73 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.143482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.144136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.144424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.144706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.163676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:12.173744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:12.214953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99cbf39-84fc-448f-9c62-a79bc239ca04 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.215365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.216825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.251540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.252338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.252624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.252903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.263826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.265374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:12.284103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:12.285054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:12.285982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:12.285982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:12.285982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:12.285982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:12.285982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 519/1046] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:12.290848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.317699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 528/1047] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:11 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 05:50:12.323662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.387482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:12.401678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac874990-89c4-42ce-be8d-d810afd9019a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.411607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:12.434970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6de3ad-7f8a-4ab6-9ef3-1ae06dac1d13 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.441043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.441727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.442415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.442706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.443470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:12.443786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.446400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.458068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 520/1048] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:12.463539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.470055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.470923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.471205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.471513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.472714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] RBAC: Authorizing `identity:get_project(project_id=6590966bb721493e886d7dec9c3481d5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.474148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.475873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 529/1049] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/projects/6590966bb721493e886d7dec9c3481d5 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:12.481664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.574891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083ca548-3477-4612-9285-5844c4ce121a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.581351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946cc8b2-5142-4b86-a1a9-d6ef169b3880 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.607335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:12.611957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.612840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.613034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.613301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.617874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.618520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.618803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.619079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.619924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=6590966bb721493e886d7dec9c3481d5, user.id=26d38509c5294b9e92c69fd03a39fa39) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:12.620247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.620800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.621692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.622220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.627192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 530/1050] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/role_assignments?user.id=26d38509c5294b9e92c69fd03a39fa39&scope.project.id=6590966bb721493e886d7dec9c3481d5&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:50:12.636993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:12.637985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:12.638916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:12.643353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 521/1051] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:12.644871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.752044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca06477-2ec8-492a-a6c5-cc83622fcd80 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.793392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.794052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.794335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.794612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.795428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:12.795739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.797163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cd02de-c310-49e2-8aca-f892aed61092 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.810053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 531/1052] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:12.816312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:12.845744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:12.921041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b120f3-a98d-49aa-9661-ed22072d0b78 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:12.954991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:12.956085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:12.956615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:12.957097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:12.958143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] RBAC: Adding query filter params (role.id=d1f46cd037874087a9a3b99bd7924499, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=ec34108c1ae24ef699265e0be32e99cf) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:12.958697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:12.960898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b44bb1e-4d07-4b84-932f-5af6c96eeecb None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:12.966883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 522/1053] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:12 2022] GET /identity/v3/role_assignments?user.id=ec34108c1ae24ef699265e0be32e99cf&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=d1f46cd037874087a9a3b99bd7924499 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:13.334807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199d0df-4871-4978-80dc-40d18622be48 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:13.335189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199d0df-4871-4978-80dc-40d18622be48 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:13.335958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199d0df-4871-4978-80dc-40d18622be48 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:13.337473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 532/1054] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:13.342508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:13.342805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:13.343089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:13.359948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:13.361150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:13.429980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 523/1055] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:13.452715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199d0df-4871-4978-80dc-40d18622be48 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:13.505546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:13.557400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199d0df-4871-4978-80dc-40d18622be48 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:13.595402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:13.596124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:13.596408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:13.596685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] PATH_INFO: `/v3/services/image` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:13.597333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:13.598698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:13.610715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:13.611878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 20 05:50:13.612749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:13.614016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 533/1056] 10.222.0.45 () {56 vars in 1105 bytes} [Mon Jun 20 05:50:13 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:13.618302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:13.642352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:13.721307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e93fa00-270e-42bb-b390-f1c9d539d1d1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:13.756514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:13.757214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:13.757495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:13.757772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:13.758482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] RBAC: Adding query filter params (name=image) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:13.758786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:13.760167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:13.764948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 524/1057] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:13.770671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:13.876922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d91c1-637c-4712-9076-d4b985f53612 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:13.889963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:13.910939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:13.911624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:13.911909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:13.912184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:13.912879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:13.913185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:13.915224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c2be60-8889-4ea3-b82c-16b04e91edd5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:13.924665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 534/1058] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:13 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:13.925675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:14.009595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:14.025833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41776c37-c4e7-40ac-8ec6-83075e2a337a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:14.060823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62439837-a87e-49bb-aa51-aff73a9f0187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:14.061501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62439837-a87e-49bb-aa51-aff73a9f0187 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:14.061792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62439837-a87e-49bb-aa51-aff73a9f0187 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:14.062074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62439837-a87e-49bb-aa51-aff73a9f0187 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:14.062828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62439837-a87e-49bb-aa51-aff73a9f0187 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:14.064243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62439837-a87e-49bb-aa51-aff73a9f0187 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:14.084309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 525/1059] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:50:13 2022] POST /identity/v3/endpoints => generated 327 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:14.657878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141573-999a-408f-8964-9a8375e1fe74 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:14.658399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141573-999a-408f-8964-9a8375e1fe74 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:14.658479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5141573-999a-408f-8964-9a8375e1fe74 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:14.659961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 535/1060] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:14.665413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:14.665828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:14.666005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:14.682653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:14.683851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:14.758573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 526/1061] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:14.780947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141573-999a-408f-8964-9a8375e1fe74 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:14.890263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5141573-999a-408f-8964-9a8375e1fe74 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:14.924460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adfc54a-cb30-4146-95b1-8973c913f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:14.925258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfc54a-cb30-4146-95b1-8973c913f22b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:14.925543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfc54a-cb30-4146-95b1-8973c913f22b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:14.925819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adfc54a-cb30-4146-95b1-8973c913f22b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:14.935751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adfc54a-cb30-4146-95b1-8973c913f22b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8adfc54a-cb30-4146-95b1-8973c913f22b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:14.943386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:14.944303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:14.945673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 536/1062] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:14.952740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:14.980670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47cd7-d0c7-4529-8546-f88f28ffa765 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:14.980968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47cd7-d0c7-4529-8546-f88f28ffa765 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:14.981250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c47cd7-d0c7-4529-8546-f88f28ffa765 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:14.982605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 537/1063] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:14.987431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:14.987793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:14.988080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.005451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:15.006647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:15.025332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.038091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.073567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65caa9fc-5c63-47a6-85da-4f16850980df None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.077837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 538/1064] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:15.100211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.108887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.109611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.109907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.110202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.110940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:15.111263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.112641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.126417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 527/1065] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:15.127431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.223719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbecbb5-ea58-4258-a448-cd80ba24e465 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.232333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbe8fe-3bc2-4ede-9a8f-d7ba045de79f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.262227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.267573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.268559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.268918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.269116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.269331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.270109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.270456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.270796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] PATH_INFO: `/v3/users/26d38509c5294b9e92c69fd03a39fa39` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.272084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] RBAC: Authorizing `identity:get_user(user_id=26d38509c5294b9e92c69fd03a39fa39)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.274104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.276038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 528/1066] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/users/26d38509c5294b9e92c69fd03a39fa39 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:15.281436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.281563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:15.292183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:15.293297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:15.294352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:15.300068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 539/1067] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:15.301786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.330692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.388456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c19ac2-8a46-46fc-b2bf-df420635577b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.407636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fc381-435f-4ff4-be5c-e421c4913c17 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.422176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.423039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.423351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.423630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.424792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] RBAC: Authorizing `identity:get_project(project_id=6590966bb721493e886d7dec9c3481d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.426206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.427913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 529/1068] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/projects/6590966bb721493e886d7dec9c3481d5 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:15.434119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.447355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.448065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.448345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.448664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.449454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:15.449760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.451147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.463824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 540/1069] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:15.465401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.505323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.541669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db731f48-c977-4edd-8bb8-798131469458 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.570328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.577175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.578099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.578386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.578664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5/users/26d38509c5294b9e92c69fd03a39fa39/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.580693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] RBAC: Authorizing `identity:create_grant(project_id=6590966bb721493e886d7dec9c3481d5, user_id=26d38509c5294b9e92c69fd03a39fa39, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.580965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32373c18-3c7e-4a54-a19b-f65440b996ba None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.582124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.597748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 530/1070] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:50:15 2022] PUT /identity/v3/projects/6590966bb721493e886d7dec9c3481d5/users/26d38509c5294b9e92c69fd03a39fa39/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 05:50:15.619204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.620140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.620496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.620841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.633974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.635605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:15.655020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:15.656311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:15.657439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:15.657439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:15.657439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:15.657439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:15.662641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 541/1071] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:15.664602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.774112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4ba3a-957f-41f5-bae1-9dae1d146107 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.800734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:15.812162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687130-97eb-412c-b512-19aec066207f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.812842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687130-97eb-412c-b512-19aec066207f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.813132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687130-97eb-412c-b512-19aec066207f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.813412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f687130-97eb-412c-b512-19aec066207f None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.814143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687130-97eb-412c-b512-19aec066207f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:15.814459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687130-97eb-412c-b512-19aec066207f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.817889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f687130-97eb-412c-b512-19aec066207f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.832609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 531/1072] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:15.838122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:15.940997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fd6708-7464-4071-a429-489595d8a3c4 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:15.974440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:15.975242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:15.975564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:15.975901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:15.984204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:15.985653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3ed39608-35ab-425c-8eea-3849443929ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:50:15.999990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:16.000974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:50:16.001936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:16.003322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 542/1073] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:50:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:16.007182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687130-97eb-412c-b512-19aec066207f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.106155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f687130-97eb-412c-b512-19aec066207f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.131822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1d63-2c27-4772-943b-5a43b16177b8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.132152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1d63-2c27-4772-943b-5a43b16177b8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.132439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd1d63-2c27-4772-943b-5a43b16177b8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.133954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 543/1074] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:16.138575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.138866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.139147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.140052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.140722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.141008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.141284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.142115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:16.142426 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.143869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.144620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.158462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:16.159776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:16.163011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 532/1075] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:16.168782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.217105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.235288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 544/1076] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:16.257759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.281406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192e9a-085b-4ccc-bbb1-9f5c434e7ebf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.316639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.317434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.317721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.318001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.328562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.330006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:16.347646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:16.348592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:16.350282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:16.350282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:16.350282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:16.350282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:16.350282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 533/1077] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:16.354810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.368730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742eb707-45cc-460e-9cd8-349e15f7a3db None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.401910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.408445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.409208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.409494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.409774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.448220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.455513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dbd84a-9570-49d7-a156-d59c35ae8782 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.478123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.479612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.490196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.491043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.491547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.491993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.492853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:16.493321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.494810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.506158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 534/1078] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:16.513048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.584700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 545/1079] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 20 05:50:16.590850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.622142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b728725-b8bb-406a-9a9b-efe092375412 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.622316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.659796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.660745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.661027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.661334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.668606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.670064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:50:16.685080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:16.686030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:16.686932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 535/1080] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:16.691905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.695644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b8dcd7-8b6f-4be9-9268-c7aac82ef22f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.712043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.732456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.733305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.733597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.733876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.744793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.746250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:16.764562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:16.765491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:16.767085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:16.767085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:16.767085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:16.767085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:16.767085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 546/1081] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:16.771923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.806867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03850f76-8652-4bba-b106-07a75059a2da None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.845245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.845900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.846179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.846494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.847323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:16.847766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.849060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.868193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 536/1082] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:16.868798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:16.870933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.890375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e8f16-0c28-4c2c-94f1-8d7fe2eb4c8e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:16.941790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:16.942465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:16.942753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:16.943034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:16.943795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:16.944112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:16.945556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4dd398-946a-46ec-8e43-5e6a75435519 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:16.967001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:16.976260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 547/1083] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:17.006366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a6c45d-1823-4c86-8145-2c2758275126 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:17.051320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d3cec-d2f9-40f0-afae-ad1232c7aad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:17.052248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d3cec-d2f9-40f0-afae-ad1232c7aad6 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:17.052537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d3cec-d2f9-40f0-afae-ad1232c7aad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:17.052818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d3cec-d2f9-40f0-afae-ad1232c7aad6 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/ec34108c1ae24ef699265e0be32e99cf/roles/d1f46cd037874087a9a3b99bd7924499` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:17.055278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d3cec-d2f9-40f0-afae-ad1232c7aad6 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=ec34108c1ae24ef699265e0be32e99cf, role_id=d1f46cd037874087a9a3b99bd7924499)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:17.056773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d3cec-d2f9-40f0-afae-ad1232c7aad6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:17.082024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 537/1084] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:50:16 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/ec34108c1ae24ef699265e0be32e99cf/roles/d1f46cd037874087a9a3b99bd7924499 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 05:50:17.648952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62d67-0c9e-4b8c-9eea-e228210c4d51 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:17.649275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62d67-0c9e-4b8c-9eea-e228210c4d51 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:17.649560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f62d67-0c9e-4b8c-9eea-e228210c4d51 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:17.650931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 548/1085] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:17.656206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:17.656508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:17.656794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:17.673777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:17.674955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:17.739337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 538/1086] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:17.762191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62d67-0c9e-4b8c-9eea-e228210c4d51 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:17.864632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f62d67-0c9e-4b8c-9eea-e228210c4d51 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:17.896573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:17.897363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:17.897674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:17.897953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:17.908110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:17.916164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:17.917069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:17.917924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 549/1087] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:17.923301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:17.986628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:18.021377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efbb9b0-9a77-4936-beda-d748a56c77eb None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:18.054251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:18.054916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:18.055206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:18.055517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:18.056260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:18.056572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:18.057929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:18.065862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 539/1088] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:18.072174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:18.075989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:18.169716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b25fa8-6a8c-4e7a-abd0-68415f7d0f3c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:18.203907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:18.204783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:18.205069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:18.205345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] PATH_INFO: `/v3/users/26d38509c5294b9e92c69fd03a39fa39` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:18.206535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] RBAC: Authorizing `identity:get_user(user_id=26d38509c5294b9e92c69fd03a39fa39)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:18.208009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:18.209652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 550/1089] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:18 2022] GET /identity/v3/users/26d38509c5294b9e92c69fd03a39fa39 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:18.215865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:18.313565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e73d31-7da1-474b-a70d-0b5ca46b5154 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:18.338725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:18.347169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:18.347846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:18.348134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:18.348416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:18.413822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:18.415267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:18.508944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 540/1090] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:18.514599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:18.582721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:18.612819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a2f1b9-fce1-404c-bab9-e827952cbda0 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:18.645522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414baee2-4a77-4e06-a839-18d2cdacd821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:18.646376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414baee2-4a77-4e06-a839-18d2cdacd821 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:18.646664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414baee2-4a77-4e06-a839-18d2cdacd821 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:18.646936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414baee2-4a77-4e06-a839-18d2cdacd821 None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:18.648764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414baee2-4a77-4e06-a839-18d2cdacd821 None admin] RBAC: Authorizing `identity:get_project(project_id=6590966bb721493e886d7dec9c3481d5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:18.650196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414baee2-4a77-4e06-a839-18d2cdacd821 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:18.651900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 551/1091] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:18 2022] GET /identity/v3/projects/6590966bb721493e886d7dec9c3481d5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:18.657970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:18.684577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:18.754371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12ad309-4e0f-4c33-8e74-b23b1f8a3da3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:18.787705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:18.788316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:18.788601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:18.788881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:18.789718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=6590966bb721493e886d7dec9c3481d5, user.id=26d38509c5294b9e92c69fd03a39fa39) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:18.790032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:18.791485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0199907-e10f-44c5-b0ca-1cb59fb3cd4d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:18.797234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 541/1092] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:18 2022] GET /identity/v3/role_assignments?user.id=26d38509c5294b9e92c69fd03a39fa39&scope.project.id=6590966bb721493e886d7dec9c3481d5&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:19.114615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387be729-fdfc-4a3a-9cee-782027f4596c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.115138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387be729-fdfc-4a3a-9cee-782027f4596c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.115138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387be729-fdfc-4a3a-9cee-782027f4596c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:19.117032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 552/1093] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:19.122280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.122686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.122992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:19.140079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:19.141267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:19.206300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 542/1094] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:19.230051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387be729-fdfc-4a3a-9cee-782027f4596c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:19.330078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387be729-fdfc-4a3a-9cee-782027f4596c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:19.363273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:19.364106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.364395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.364745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:19.372600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:19.376006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:19.384336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:19.385446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:19.386435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 553/1095] 10.222.0.45 () {56 vars in 1120 bytes} [Mon Jun 20 05:50:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:19.391535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:19.415111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:19.488190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ada96-732c-47dc-8dac-8019c0ec2bcf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:19.520956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:19.521626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.521913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.522188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:19.522980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:19.523314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:19.524671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:19.532387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 543/1096] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:50:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:19.538658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:19.637203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46771c81-1387-412f-89a4-fb76c13f37d3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:19.670216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:19.670999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.671307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.671585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:19.682673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:19.684134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:19.700651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:19.701519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:19.702421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:19.702421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:19.702421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:19.702421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:19.702421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 554/1097] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:19.707713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:19.789455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:19.805415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2692e9-20cc-4924-97be-a1e95cb0682a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:19.840079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:19.840739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.841030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.841305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:19.842025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:19.842335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:19.843977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:19.856205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 544/1098] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:19.861673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:19.914868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:19.963234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d194361-c0d1-41cb-b36f-c8e01987b7b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:19.995814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:19.996598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:19.996884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:19.997162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.004387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:20.005831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:50:20.019471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:20.020598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:20.021560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 555/1099] 10.222.0.45 () {56 vars in 1093 bytes} [Mon Jun 20 05:50:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:20.027027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:20.122145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6369c7-37ed-483e-b71c-fe3d14ba8ffc None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:20.155823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:20.156484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.156771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.157050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.157875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:20.158186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:20.159612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:20.176053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 545/1100] 10.222.0.45 () {56 vars in 1133 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:20.181701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:20.278620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8657bfcd-8303-4076-8c49-d812eb86aafe None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:20.291028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:20.311867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:20.312512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.312798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.313075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.377851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:20.379329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:20.474396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 556/1101] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:20.480196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:20.483038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:20.580328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f6aa-7d6c-4af6-8507-f60ae3287cd8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:20.613402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:20.614193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.614476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.614749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.625031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:20.626466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:20.643962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:20.644995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:20.646064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:50:20.646064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:20.646064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:50:20.646064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:20.647401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 546/1102] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:20.650867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:20.664779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:20.747441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c288c-bc61-4bb6-9d30-54459697a442 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:20.780242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:20.780896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.781180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.781454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.782193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:20.782500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:20.783904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a618992f-d26c-4fb4-af60-9792e8cf4fb7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:20.795265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 557/1103] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:20.800872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:20.840068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e626a3c-07a1-4b8a-aee6-4b9d1e2cc1d4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.840360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e626a3c-07a1-4b8a-aee6-4b9d1e2cc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.840641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e626a3c-07a1-4b8a-aee6-4b9d1e2cc1d4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.842002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 558/1104] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:50:20.847113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.847442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.847725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.861014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:20.864531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:20.865722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:20.905548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a16b7-ce90-4606-9959-55306b403ff2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:20.931861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 559/1105] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 20 05:50:20.939387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:20.940173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:20.940456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:20.940731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:20.947133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:20.948581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:20.956367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:20.958387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:50:20.962158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:20.963350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:20.964460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 547/1106] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:20.969195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:21.060835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7da52-283c-437f-be59-bdfcd0c86322 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:21.066567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07b6c5a-7e07-4759-9bd2-87e227f43b19 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:21.079757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:21.093999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:21.094795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:21.095079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:21.095475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:21.096647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:21.098078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:21.099722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 560/1107] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:21.102390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:21.103162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:21.103542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:21.103880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:21.104736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:21.105054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:21.105850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:21.106450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:21.119059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 548/1108] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:50:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:21.125167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:21.172749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:21.201740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52610f-1ef8-4c0c-a4dc-53969ef2be5c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:21.222704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b127c59-6796-4c4f-8866-cf883a63a0f1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:21.236310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afe3823-804e-433a-9679-9656b946bb0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:21.236975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3823-804e-433a-9679-9656b946bb0f None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:21.237264 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3823-804e-433a-9679-9656b946bb0f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:21.237545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afe3823-804e-433a-9679-9656b946bb0f None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:21.238245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe3823-804e-433a-9679-9656b946bb0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:21.239721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afe3823-804e-433a-9679-9656b946bb0f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:21.256497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:21.257175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:21.257462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:21.257736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:21.258569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] RBAC: Adding query filter params (role.id=d1f46cd037874087a9a3b99bd7924499, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=ec34108c1ae24ef699265e0be32e99cf) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:21.258695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 561/1109] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:50:21 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:50:21.258882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:21.260322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdbea9-4507-46fe-93f4-fe3b87608680 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:21.267302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 549/1110] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:21 2022] GET /identity/v3/role_assignments?user.id=ec34108c1ae24ef699265e0be32e99cf&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=d1f46cd037874087a9a3b99bd7924499 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:50:23.323161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea71acb-f04b-49fb-9b83-72465fcf9f7b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:23.323761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea71acb-f04b-49fb-9b83-72465fcf9f7b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:23.323761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea71acb-f04b-49fb-9b83-72465fcf9f7b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:23.325150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 562/1111] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:23.330471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:23.330915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:23.331022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:23.348026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:23.349204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:23.377086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:23.413319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 550/1112] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:23.437140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea71acb-f04b-49fb-9b83-72465fcf9f7b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:23.534403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea71acb-f04b-49fb-9b83-72465fcf9f7b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:23.558600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:23.567690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:23.568529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:23.568813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:23.569138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] PATH_INFO: `/v3/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:23.570587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] RBAC: Authorizing `identity:get_role(role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:23.571977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:23.573348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 563/1113] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:23 2022] GET /identity/v3/roles/df14633e5004446c8c52b442c7d11a91 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:23.578910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:23.677273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66437697-a859-4c66-9a36-fdf5d7b41107 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:23.709213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:23.710583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:23.710868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:23.711143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] PATH_INFO: `/v3/users/3a1c5a70530947f5be691c0db25d513c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:23.723816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] RBAC: Authorizing `identity:get_user(user_id=3a1c5a70530947f5be691c0db25d513c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:23.725312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:23.727478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 551/1114] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:23 2022] GET /identity/v3/users/3a1c5a70530947f5be691c0db25d513c => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:23.733423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:23.830259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aad635-bc82-43dc-98d1-84477dc1a470 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:23.862754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:23.863420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:23.863707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:23.863987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:23.921101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:23.928560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:23.930000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:24.023689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 564/1115] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:24.029538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:24.055492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:24.126691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef9650-bc78-40cb-bf7c-f014efa88a8f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:24.160048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1ccd9-7b6a-4439-bdc0-20141d21cdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:24.160901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1ccd9-7b6a-4439-bdc0-20141d21cdaa None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:24.161187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1ccd9-7b6a-4439-bdc0-20141d21cdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:24.161469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1ccd9-7b6a-4439-bdc0-20141d21cdaa None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:24.162638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1ccd9-7b6a-4439-bdc0-20141d21cdaa None admin] RBAC: Authorizing `identity:get_project(project_id=6590966bb721493e886d7dec9c3481d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:24.164093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a1ccd9-7b6a-4439-bdc0-20141d21cdaa None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:24.165790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 552/1116] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:24 2022] GET /identity/v3/projects/6590966bb721493e886d7dec9c3481d5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:24.171831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:24.260178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:24.266732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b551bd-14e2-440b-ba46-0f5e25de11aa None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:24.298201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c416260-38dc-4393-963b-f19238512445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:24.298828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c416260-38dc-4393-963b-f19238512445 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:24.299109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c416260-38dc-4393-963b-f19238512445 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:24.299410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c416260-38dc-4393-963b-f19238512445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:24.300219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c416260-38dc-4393-963b-f19238512445 None admin] RBAC: Adding query filter params (role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=6590966bb721493e886d7dec9c3481d5, user.id=3a1c5a70530947f5be691c0db25d513c) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:24.300529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c416260-38dc-4393-963b-f19238512445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:24.301937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c416260-38dc-4393-963b-f19238512445 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:24.307702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 565/1117] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:24 2022] GET /identity/v3/role_assignments?user.id=3a1c5a70530947f5be691c0db25d513c&scope.project.id=6590966bb721493e886d7dec9c3481d5&role.id=df14633e5004446c8c52b442c7d11a91 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:26.276815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40f080-d4f7-4fa6-9326-ccb16f059d4a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:26.277362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40f080-d4f7-4fa6-9326-ccb16f059d4a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:26.277362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d40f080-d4f7-4fa6-9326-ccb16f059d4a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:26.278727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 553/1118] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:26.283970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:26.284352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:26.284539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:26.301231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:26.302403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:26.366531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 566/1119] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:26.389135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40f080-d4f7-4fa6-9326-ccb16f059d4a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:26.480196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:26.488295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d40f080-d4f7-4fa6-9326-ccb16f059d4a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:26.519838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:26.520678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:26.521030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:26.521312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] PATH_INFO: `/v3/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:26.523319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] RBAC: Authorizing `identity:get_role(role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:26.524672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:26.526008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 554/1120] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:26 2022] GET /identity/v3/roles/df14633e5004446c8c52b442c7d11a91 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:26.531857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:26.614984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:26.627137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b175b8-1fe4-4e1f-85cb-d609d9152f0e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:26.659783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:26.660646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:26.660934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:26.661214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] PATH_INFO: `/v3/users/3a1c5a70530947f5be691c0db25d513c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:26.662393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] RBAC: Authorizing `identity:get_user(user_id=3a1c5a70530947f5be691c0db25d513c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:26.664482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:26.666107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 567/1121] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:26 2022] GET /identity/v3/users/3a1c5a70530947f5be691c0db25d513c => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:26.671726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:26.765943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14c281-15d8-474b-a62d-759ff9c46485 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:26.799281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf5e76-9b41-4197-a8ad-4a3c7be4175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:26.800071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5e76-9b41-4197-a8ad-4a3c7be4175b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:26.800357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5e76-9b41-4197-a8ad-4a3c7be4175b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:26.800635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf5e76-9b41-4197-a8ad-4a3c7be4175b None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:26.801800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5e76-9b41-4197-a8ad-4a3c7be4175b None admin] RBAC: Authorizing `identity:get_project(project_id=6590966bb721493e886d7dec9c3481d5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:26.803207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bf5e76-9b41-4197-a8ad-4a3c7be4175b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:26.804892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 555/1122] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:26 2022] GET /identity/v3/projects/6590966bb721493e886d7dec9c3481d5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:26.810490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:26.907596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445870ef-e16b-4fde-b1ef-be8f06611fea None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:26.939654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724123c-ef4a-445b-9a4f-5f0ee7382bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:26.940518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724123c-ef4a-445b-9a4f-5f0ee7382bf0 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:26.940800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724123c-ef4a-445b-9a4f-5f0ee7382bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:26.941074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724123c-ef4a-445b-9a4f-5f0ee7382bf0 None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5/users/3a1c5a70530947f5be691c0db25d513c/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:26.943226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724123c-ef4a-445b-9a4f-5f0ee7382bf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6590966bb721493e886d7dec9c3481d5, user_id=3a1c5a70530947f5be691c0db25d513c, role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:26.945186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8724123c-ef4a-445b-9a4f-5f0ee7382bf0 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:26.957599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 568/1123] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:50:26 2022] PUT /identity/v3/projects/6590966bb721493e886d7dec9c3481d5/users/3a1c5a70530947f5be691c0db25d513c/roles/df14633e5004446c8c52b442c7d11a91 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:50:28.932927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb3f04-4625-40ba-989e-fd9b928db6c5 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:28.933467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb3f04-4625-40ba-989e-fd9b928db6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:28.933467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb3f04-4625-40ba-989e-fd9b928db6c5 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:28.934835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 556/1124] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:28.940041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:28.940469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:28.940577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:28.957296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:28.958462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:28.968984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:29.024459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 569/1125] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:29.048126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb3f04-4625-40ba-989e-fd9b928db6c5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:29.077799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:29.148831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb3f04-4625-40ba-989e-fd9b928db6c5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:29.180260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:29.181098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:29.181379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:29.181651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] PATH_INFO: `/v3/roles/df14633e5004446c8c52b442c7d11a91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:29.183097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] RBAC: Authorizing `identity:get_role(role_id=df14633e5004446c8c52b442c7d11a91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:29.184463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:29.185812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 557/1126] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:29 2022] GET /identity/v3/roles/df14633e5004446c8c52b442c7d11a91 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:29.191498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:29.286266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db794f51-f7f5-4354-82f5-61077b949555 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:29.317910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:29.319312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:29.319597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:29.319871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] PATH_INFO: `/v3/users/3a1c5a70530947f5be691c0db25d513c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:29.321035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] RBAC: Authorizing `identity:get_user(user_id=3a1c5a70530947f5be691c0db25d513c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:29.322566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:29.324210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 570/1127] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:29 2022] GET /identity/v3/users/3a1c5a70530947f5be691c0db25d513c => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:29.329846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:29.422793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:29.426988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25e89d-9ac0-43b5-834c-b33c21f1b3f5 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:29.460617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:29.461260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:29.461548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:29.461826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:29.525680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:29.527108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:29.614857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:29.620064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 558/1128] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:29.625842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:29.641507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:29.722510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906543a-64e4-4b7a-a5d2-bc2a60f92f39 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:29.754703 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972eba8-ac88-4917-a00f-6f91f677085d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:29.755581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972eba8-ac88-4917-a00f-6f91f677085d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:29.755869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972eba8-ac88-4917-a00f-6f91f677085d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:29.756724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972eba8-ac88-4917-a00f-6f91f677085d None admin] PATH_INFO: `/v3/projects/6590966bb721493e886d7dec9c3481d5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:29.757895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972eba8-ac88-4917-a00f-6f91f677085d None admin] RBAC: Authorizing `identity:get_project(project_id=6590966bb721493e886d7dec9c3481d5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:29.759349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972eba8-ac88-4917-a00f-6f91f677085d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:29.761020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 571/1129] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:29 2022] GET /identity/v3/projects/6590966bb721493e886d7dec9c3481d5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:29.766931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:29.861669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa775020-47c8-42cb-b51d-a9cbf17a4899 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:29.894268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:29.894849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:29.895135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:29.895441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:29.896259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] RBAC: Adding query filter params (role.id=df14633e5004446c8c52b442c7d11a91, scope.project.id=6590966bb721493e886d7dec9c3481d5, user.id=3a1c5a70530947f5be691c0db25d513c) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:29.896570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:29.897993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d900b-70e9-401b-b86a-b15bc68a6b29 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:29.903617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 559/1130] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:29 2022] GET /identity/v3/role_assignments?user.id=3a1c5a70530947f5be691c0db25d513c&scope.project.id=6590966bb721493e886d7dec9c3481d5&role.id=df14633e5004446c8c52b442c7d11a91 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:31.936993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12772c-c257-425d-b59d-e27b87568a47 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:31.937496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12772c-c257-425d-b59d-e27b87568a47 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:31.937603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12772c-c257-425d-b59d-e27b87568a47 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:31.938903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 572/1131] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:31.944480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:31.944967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:31.945067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:31.961143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:31.962317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:32.015496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:32.025542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 560/1132] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:32.048378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12772c-c257-425d-b59d-e27b87568a47 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:32.128266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:32.146607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12772c-c257-425d-b59d-e27b87568a47 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:32.179057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c32826-7ddb-4c56-a55f-546ee886b53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:32.179865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c32826-7ddb-4c56-a55f-546ee886b53f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:32.180153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c32826-7ddb-4c56-a55f-546ee886b53f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:32.180432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c32826-7ddb-4c56-a55f-546ee886b53f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:32.181588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c32826-7ddb-4c56-a55f-546ee886b53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:32.183021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c32826-7ddb-4c56-a55f-546ee886b53f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:32.184629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 573/1133] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:32.190360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:32.284439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90557d3e-d998-4c2c-b0d3-481c5946ae8d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:32.315706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f8490c-a8bc-42d8-9bcd-2e160ad4c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:32.316342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8490c-a8bc-42d8-9bcd-2e160ad4c106 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:32.316623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8490c-a8bc-42d8-9bcd-2e160ad4c106 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:32.316897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f8490c-a8bc-42d8-9bcd-2e160ad4c106 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:32.317593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8490c-a8bc-42d8-9bcd-2e160ad4c106 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:32.319544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f8490c-a8bc-42d8-9bcd-2e160ad4c106 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:32.343334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 561/1134] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:50:32 2022] POST /identity/v3/projects => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:34.379907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499a15b-cb4d-4852-9a31-c08f16ac3522 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:34.380456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499a15b-cb4d-4852-9a31-c08f16ac3522 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:34.380456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499a15b-cb4d-4852-9a31-c08f16ac3522 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:34.382319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 574/1135] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:34.387636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:34.388063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:34.388193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:34.404941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:34.406109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:34.442911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:34.470291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 562/1136] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:34.494213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499a15b-cb4d-4852-9a31-c08f16ac3522 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:34.591801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499a15b-cb4d-4852-9a31-c08f16ac3522 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:34.623936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe99c9-c831-4a0c-a1d3-1d2da23df620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:34.624710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe99c9-c831-4a0c-a1d3-1d2da23df620 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:34.624995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe99c9-c831-4a0c-a1d3-1d2da23df620 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:34.625271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe99c9-c831-4a0c-a1d3-1d2da23df620 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:34.626426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe99c9-c831-4a0c-a1d3-1d2da23df620 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:34.627895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe99c9-c831-4a0c-a1d3-1d2da23df620 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:34.629474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 575/1137] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:50:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:34.635567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:34.729100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f366516-1e1e-45f3-9b50-2c5bd6013ab1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:34.761439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb67c2-3044-4aff-9d4b-fd1deaf8edd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:34.762091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb67c2-3044-4aff-9d4b-fd1deaf8edd9 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:34.762375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb67c2-3044-4aff-9d4b-fd1deaf8edd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:34.762649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb67c2-3044-4aff-9d4b-fd1deaf8edd9 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:34.763369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb67c2-3044-4aff-9d4b-fd1deaf8edd9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:34.764780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb67c2-3044-4aff-9d4b-fd1deaf8edd9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:34.783090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 563/1138] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:50:34 2022] POST /identity/v3/users => generated 286 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:36.837632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73418-c992-4057-9d63-80af780b1478 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:36.838071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73418-c992-4057-9d63-80af780b1478 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:36.838194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d73418-c992-4057-9d63-80af780b1478 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:36.839595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 576/1139] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:36.845059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:36.845463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:36.845625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:36.861815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:36.862997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:36.885957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:36.928261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 564/1140] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:36.951886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73418-c992-4057-9d63-80af780b1478 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:36.972857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:37.052313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d73418-c992-4057-9d63-80af780b1478 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:37.084591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:37.085309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:37.085590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:37.085862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:37.095824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:37.103912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:37.105191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:37.106432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 577/1141] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:37.110903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:37.205074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7568d3-266e-4d0e-9820-f85fe14c8838 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:37.238364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:37.239021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:37.239336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:37.239622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:37.240390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:37.240701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:37.242057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:37.249782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 565/1142] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:37.255273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:37.330587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:37.352201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e156f-43e9-4bf0-a9eb-9e835bb07b2f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:37.383819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:37.384669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:37.384949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:37.385287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] PATH_INFO: `/v3/users/967c8aac05d74fb4a8d16043171b6188` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:37.398282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] RBAC: Authorizing `identity:get_user(user_id=967c8aac05d74fb4a8d16043171b6188)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:37.399743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:37.401333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 578/1143] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:37 2022] GET /identity/v3/users/967c8aac05d74fb4a8d16043171b6188 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:37.407021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:37.431078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:37.503381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f89726-6a2d-4942-bd83-ee99cd2a0bb7 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:37.535946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:37.536582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:37.536892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:37.537180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:37.603060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:37.604463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:37.626143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:37.699040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 566/1144] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:37.704831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:37.799945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb00a74-35de-465c-9de5-3fb1794e9825 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:37.834538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7adb6a-e33b-4011-8e34-5056bbd94a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:37.835429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7adb6a-e33b-4011-8e34-5056bbd94a70 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:37.835724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7adb6a-e33b-4011-8e34-5056bbd94a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:37.835985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7adb6a-e33b-4011-8e34-5056bbd94a70 None admin] PATH_INFO: `/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:37.837155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7adb6a-e33b-4011-8e34-5056bbd94a70 None admin] RBAC: Authorizing `identity:get_project(project_id=e10a3fe98cef4c49b824db6df67ca9e4)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:37.838594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7adb6a-e33b-4011-8e34-5056bbd94a70 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:37.840307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 579/1145] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:37 2022] GET /identity/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:37.846554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:37.942111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e81940-ac39-463b-82d2-d22078aa708f None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:37.964320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:37.976417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:37.977062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:37.977353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:37.977634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:37.978464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=e10a3fe98cef4c49b824db6df67ca9e4, user.id=967c8aac05d74fb4a8d16043171b6188) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:37.978787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:37.980233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc2277-5d44-47b6-8048-25e8a25229db None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:37.985552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 567/1146] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:37 2022] GET /identity/v3/role_assignments?user.id=967c8aac05d74fb4a8d16043171b6188&scope.project.id=e10a3fe98cef4c49b824db6df67ca9e4&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:39.967001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157f57-3fe5-4d9b-aa66-b43a11a1b73f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:39.967552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157f57-3fe5-4d9b-aa66-b43a11a1b73f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:39.967662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa157f57-3fe5-4d9b-aa66-b43a11a1b73f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:39.969212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 580/1147] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:39.974989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:39.975404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:39.975594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:39.991825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:39.993014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:40.055957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 568/1148] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:40.078310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157f57-3fe5-4d9b-aa66-b43a11a1b73f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:40.082052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:40.175297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa157f57-3fe5-4d9b-aa66-b43a11a1b73f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:40.206588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:40.207387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:40.207673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:40.207948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:40.218306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:40.226394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:40.227482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:40.228472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 581/1149] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:40.233521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:40.328148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44804c5-efbb-465e-b50b-da2d8bea47f9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:40.344627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:40.362053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:40.362711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:40.362999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:40.363314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:40.364058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:40.364371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:40.365751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:40.373555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 569/1150] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:40.379108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:40.438944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:40.476221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b2de0-c2c3-4daa-b734-c6de8d524bc7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:40.507809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:40.508658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:40.508941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:40.509213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] PATH_INFO: `/v3/users/967c8aac05d74fb4a8d16043171b6188` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:40.510367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] RBAC: Authorizing `identity:get_user(user_id=967c8aac05d74fb4a8d16043171b6188)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:40.511817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:40.513385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 582/1151] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:40 2022] GET /identity/v3/users/967c8aac05d74fb4a8d16043171b6188 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:40.518815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:40.613767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884ffc88-2df4-4a0c-abfa-f1646ff652b2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:40.645437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8b035d-2328-4da7-b8d8-bcbbe2d030d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:40.646336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b035d-2328-4da7-b8d8-bcbbe2d030d2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:40.646622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b035d-2328-4da7-b8d8-bcbbe2d030d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:40.646901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8b035d-2328-4da7-b8d8-bcbbe2d030d2 None admin] PATH_INFO: `/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:40.648086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b035d-2328-4da7-b8d8-bcbbe2d030d2 None admin] RBAC: Authorizing `identity:get_project(project_id=e10a3fe98cef4c49b824db6df67ca9e4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:40.649505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8b035d-2328-4da7-b8d8-bcbbe2d030d2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:40.651157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 570/1152] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:40 2022] GET /identity/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:40.656672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:40.751981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2216f8-8393-466d-a12b-66f26510333b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:40.779585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:40.784923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2cb3fe-37cf-4da9-b217-3d02ca0d98d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:40.785817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb3fe-37cf-4da9-b217-3d02ca0d98d7 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:40.786100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb3fe-37cf-4da9-b217-3d02ca0d98d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:40.786376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2cb3fe-37cf-4da9-b217-3d02ca0d98d7 None admin] PATH_INFO: `/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4/users/967c8aac05d74fb4a8d16043171b6188/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:40.788344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cb3fe-37cf-4da9-b217-3d02ca0d98d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e10a3fe98cef4c49b824db6df67ca9e4, user_id=967c8aac05d74fb4a8d16043171b6188, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:40.789765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2cb3fe-37cf-4da9-b217-3d02ca0d98d7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:40.803244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 583/1153] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:50:40 2022] PUT /identity/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4/users/967c8aac05d74fb4a8d16043171b6188/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:50:42.778366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250e464-0573-4848-a716-bcf433132e12 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:42.778914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250e464-0573-4848-a716-bcf433132e12 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:42.778914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d250e464-0573-4848-a716-bcf433132e12 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:42.780316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 571/1154] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:42.785316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:42.785736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:42.785875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:42.802235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:42.803434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:42.867460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 584/1155] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:42.891081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250e464-0573-4848-a716-bcf433132e12 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:42.937567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:42.989546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d250e464-0573-4848-a716-bcf433132e12 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:43.021759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:43.022530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:43.022816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:43.023091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:43.033231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:43.040688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:43.041765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:43.042806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 572/1156] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:43.047859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:43.142409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0125e5-39e1-48be-a1e6-b32cb3c104cd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:43.163651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:43.176104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:43.176769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:43.177059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:43.177844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:43.178590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:43.178905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:43.180316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:43.188176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 585/1157] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:43.194302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:43.290456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813eabfc-dbaa-4c7d-be8d-9112bf906c35 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:43.297277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:43.324155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:43.325013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:43.325300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:43.325579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] PATH_INFO: `/v3/users/967c8aac05d74fb4a8d16043171b6188` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:43.326754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] RBAC: Authorizing `identity:get_user(user_id=967c8aac05d74fb4a8d16043171b6188)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:43.328200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:43.329813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 573/1158] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:43 2022] GET /identity/v3/users/967c8aac05d74fb4a8d16043171b6188 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:43.335462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:43.433497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164d77ea-af6f-4b6e-bcc2-3d688ceb448f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:43.465618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:43.466257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:43.466545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:43.466823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:43.516557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:43.532068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:43.533516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:43.628850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 586/1159] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:43.635045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:43.729663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc38ec6-ed57-4f2b-b875-0e90e8b1e29d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:43.762875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af7804-42b4-4594-8736-c1844941780e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:43.763831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7804-42b4-4594-8736-c1844941780e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:43.764119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7804-42b4-4594-8736-c1844941780e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:43.764397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af7804-42b4-4594-8736-c1844941780e None admin] PATH_INFO: `/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:43.765566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af7804-42b4-4594-8736-c1844941780e None admin] RBAC: Authorizing `identity:get_project(project_id=e10a3fe98cef4c49b824db6df67ca9e4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:43.766986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af7804-42b4-4594-8736-c1844941780e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:43.768689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 574/1160] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:43 2022] GET /identity/v3/projects/e10a3fe98cef4c49b824db6df67ca9e4 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:43.774573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:43.857323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:43.873216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27648930-3390-4f0a-b14b-a7765e88d43e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:43.906119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:43.906707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:43.906994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:43.907298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:43.908134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=e10a3fe98cef4c49b824db6df67ca9e4, user.id=967c8aac05d74fb4a8d16043171b6188) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:43.908453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:43.909883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febebe2a-ff9a-4d57-832f-7d6a6b1680c1 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:43.916221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 587/1161] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:43 2022] GET /identity/v3/role_assignments?user.id=967c8aac05d74fb4a8d16043171b6188&scope.project.id=e10a3fe98cef4c49b824db6df67ca9e4&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:45.943554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93025-0cb8-42b0-a71d-baa721ee10e7 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:45.944080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93025-0cb8-42b0-a71d-baa721ee10e7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:45.944178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd93025-0cb8-42b0-a71d-baa721ee10e7 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:45.945478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 575/1162] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:45.950683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:45.951122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:45.951242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:45.969303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:45.970513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:46.035370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 588/1163] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:46.057835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd93025-0cb8-42b0-a71d-baa721ee10e7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:46.100962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:46.155431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd93025-0cb8-42b0-a71d-baa721ee10e7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:46.187228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:46.187885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:46.188167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:46.188444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:46.251671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:46.253101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:46.292256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:46.345901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 576/1164] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:46.352374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:46.425458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:46.446441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5afc5fc-1785-4fb1-b64d-0510586a7fb9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:46.478916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:46.479722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:46.480005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:46.480279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:46.486632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:46.488092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-18163a4f-615b-4bde-a163-2affc5596f4d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:46.502215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:46.503426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:50:46.504620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 589/1165] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:50:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:50:46.509113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:46.602161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bc3ee-e958-4ce7-babb-1738ac0b29fd None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:46.635013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:46.635613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:46.635896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:46.636170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:46.636885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:46.637187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:46.638521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db2466-1d37-423b-b978-cb7d276e594e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:46.645313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 577/1166] 10.222.0.45 () {56 vars in 1115 bytes} [Mon Jun 20 05:50:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:48.618199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44bb0f-7ee7-490a-a615-c0150eff5c14 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:48.618766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44bb0f-7ee7-490a-a615-c0150eff5c14 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:48.618766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44bb0f-7ee7-490a-a615-c0150eff5c14 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:48.620220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 590/1167] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:48.625213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3c5bc-60ec-4089-87f9-5bf7537253e3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:48.625627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3c5bc-60ec-4089-87f9-5bf7537253e3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:48.625767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3c5bc-60ec-4089-87f9-5bf7537253e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:48.637476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:48.642776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a4c3c5bc-60ec-4089-87f9-5bf7537253e3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:48.643986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a4c3c5bc-60ec-4089-87f9-5bf7537253e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:48.707367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 578/1168] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:48.730428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44bb0f-7ee7-490a-a615-c0150eff5c14 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:48.828201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44bb0f-7ee7-490a-a615-c0150eff5c14 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:48.860846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007954c9-da90-4093-b09c-67b9d676d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:48.861493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954c9-da90-4093-b09c-67b9d676d49a None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:48.861780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954c9-da90-4093-b09c-67b9d676d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:48.862059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954c9-da90-4093-b09c-67b9d676d49a None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:48.862798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007954c9-da90-4093-b09c-67b9d676d49a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:48.864189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007954c9-da90-4093-b09c-67b9d676d49a None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:48.872383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:48.880559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 591/1169] 10.222.0.45 () {60 vars in 1138 bytes} [Mon Jun 20 05:50:48 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:50.896689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ee3ca-f587-4ba9-880c-7925b5dc0c26 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:50.897197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ee3ca-f587-4ba9-880c-7925b5dc0c26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:50.897291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848ee3ca-f587-4ba9-880c-7925b5dc0c26 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:50.898615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 579/1170] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:50.903690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951cb5a-050c-432f-828e-03658964a55b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:50.904080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951cb5a-050c-432f-828e-03658964a55b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:50.904262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7951cb5a-050c-432f-828e-03658964a55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:50.921240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7951cb5a-050c-432f-828e-03658964a55b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:50.922419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7951cb5a-050c-432f-828e-03658964a55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:50.986309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 592/1171] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:51.010206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ee3ca-f587-4ba9-880c-7925b5dc0c26 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:51.107110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848ee3ca-f587-4ba9-880c-7925b5dc0c26 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:51.139459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c639c638-c0c9-4191-b8ac-cc27cd3f69ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:51.140307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639c638-c0c9-4191-b8ac-cc27cd3f69ba None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:51.140591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639c638-c0c9-4191-b8ac-cc27cd3f69ba None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:51.140870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c639c638-c0c9-4191-b8ac-cc27cd3f69ba None admin] PATH_INFO: `/v3/domains/d1ffd5afb0214320acbfbc6d8771b049` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:51.143519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:51.152268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639c638-c0c9-4191-b8ac-cc27cd3f69ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1ffd5afb0214320acbfbc6d8771b049)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:51.153698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c639c638-c0c9-4191-b8ac-cc27cd3f69ba None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:51.155326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 580/1172] 10.222.0.45 () {56 vars in 1183 bytes} [Mon Jun 20 05:50:51 2022] GET /identity/v3/domains/d1ffd5afb0214320acbfbc6d8771b049 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:51.160880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951cb5a-050c-432f-828e-03658964a55b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:51.256017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7951cb5a-050c-432f-828e-03658964a55b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:51.280979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:51.290073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab284-7205-4916-bfbb-2ec5c2d25251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:51.290726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab284-7205-4916-bfbb-2ec5c2d25251 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:51.291013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab284-7205-4916-bfbb-2ec5c2d25251 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:51.291316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab284-7205-4916-bfbb-2ec5c2d25251 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:51.292037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ab284-7205-4916-bfbb-2ec5c2d25251 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:51.293473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985ab284-7205-4916-bfbb-2ec5c2d25251 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:51.319301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 593/1173] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:50:51 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 05:50:53.344625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7228ac-b279-4ed7-bacb-a838a243b3cc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:53.345151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7228ac-b279-4ed7-bacb-a838a243b3cc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:53.345243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7228ac-b279-4ed7-bacb-a838a243b3cc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:53.346543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 581/1174] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:53.351756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:53.352194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:53.352325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:53.368732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:53.369919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:53.434747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 594/1175] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:53.459603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7228ac-b279-4ed7-bacb-a838a243b3cc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:53.557903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7228ac-b279-4ed7-bacb-a838a243b3cc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:53.596371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada08928-bbbd-44df-bd64-e5e981032602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:53.597163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08928-bbbd-44df-bd64-e5e981032602 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:53.597444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08928-bbbd-44df-bd64-e5e981032602 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:53.597720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada08928-bbbd-44df-bd64-e5e981032602 None admin] PATH_INFO: `/v3/domains/d1ffd5afb0214320acbfbc6d8771b049` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:53.598932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada08928-bbbd-44df-bd64-e5e981032602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d1ffd5afb0214320acbfbc6d8771b049)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:53.600396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada08928-bbbd-44df-bd64-e5e981032602 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:53.601973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 582/1176] 10.222.0.45 () {56 vars in 1183 bytes} [Mon Jun 20 05:50:53 2022] GET /identity/v3/domains/d1ffd5afb0214320acbfbc6d8771b049 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:53.607699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:53.702973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793149b-6baf-4878-b696-b1bad0a62a41 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:53.713297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:53.737330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b36635-10a9-43cd-9151-d482a5788859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:53.737991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b36635-10a9-43cd-9151-d482a5788859 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:53.738279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b36635-10a9-43cd-9151-d482a5788859 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:53.738559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b36635-10a9-43cd-9151-d482a5788859 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:53.739836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b36635-10a9-43cd-9151-d482a5788859 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:53.741264 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b36635-10a9-43cd-9151-d482a5788859 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:53.760793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 595/1177] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:50:53 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:50:55.787544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566b3ff-7b54-4a9c-8cfc-055e7ed744bc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:55.788096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566b3ff-7b54-4a9c-8cfc-055e7ed744bc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:55.788096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0566b3ff-7b54-4a9c-8cfc-055e7ed744bc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:55.789443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 583/1178] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:55.794596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:55.795056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:55.795150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:55.812205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:55.813392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:55.878067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 596/1179] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:55.901834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566b3ff-7b54-4a9c-8cfc-055e7ed744bc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:55.961775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:55.998944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0566b3ff-7b54-4a9c-8cfc-055e7ed744bc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:56.030348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:56.031122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:56.031447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:56.031725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:56.042267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:56.049652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:56.050733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:56.052508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 584/1180] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:56.057197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:56.146818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:56.153785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2feded-5146-4457-b426-f589412d6e28 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:56.187412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:56.188064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:56.188351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:56.188629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:56.189372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:56.189683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:56.191055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:56.198834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 597/1181] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:56.204295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:56.299046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861473f3-3ef2-40b9-b77e-47b18eb7a4d3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:56.321541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:56.331964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:56.332844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:56.333125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:56.333399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] PATH_INFO: `/v3/users/d091111ae2724958b0bfe4a7db800946` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:56.346850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] RBAC: Authorizing `identity:get_user(user_id=d091111ae2724958b0bfe4a7db800946)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:56.348823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:56.350419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 585/1182] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:56 2022] GET /identity/v3/users/d091111ae2724958b0bfe4a7db800946 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:56.356073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:56.452424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714a6da-6f61-4d55-b393-0d7df38deaf3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:56.485311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:56.485944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:56.486229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:56.486509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:56.511724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:56.551796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:56.553178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:56.648663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 598/1183] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:50:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:50:56.654559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:56.747977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17cbdb-8aa3-446c-96fe-a8e1d08d6d40 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:56.779732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c354c28-60f5-4da0-ab66-515529220fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:56.780565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354c28-60f5-4da0-ab66-515529220fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:56.780850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354c28-60f5-4da0-ab66-515529220fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:56.781122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c354c28-60f5-4da0-ab66-515529220fb0 None admin] PATH_INFO: `/v3/projects/381a33da6ede46b5a43b730be06d5db0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:56.782273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354c28-60f5-4da0-ab66-515529220fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=381a33da6ede46b5a43b730be06d5db0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:56.783699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c354c28-60f5-4da0-ab66-515529220fb0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:56.785329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 586/1184] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:56 2022] GET /identity/v3/projects/381a33da6ede46b5a43b730be06d5db0 => generated 361 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:56.791621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:56.846671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:56.886055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c694ee1-40d7-4248-9e93-4ee083fc3e14 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:56.917979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:56.918615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:56.918897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:56.919171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:56.920019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=381a33da6ede46b5a43b730be06d5db0, user.id=d091111ae2724958b0bfe4a7db800946) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:56.920328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:56.921736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd872f-8647-4091-a378-23eecafe2ed3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:56.926930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 599/1185] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:50:56 2022] GET /identity/v3/role_assignments?user.id=d091111ae2724958b0bfe4a7db800946&scope.project.id=381a33da6ede46b5a43b730be06d5db0&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:58.898995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086f77-e90c-4f59-af6c-b460d7327652 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:58.899505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086f77-e90c-4f59-af6c-b460d7327652 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:58.899653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c086f77-e90c-4f59-af6c-b460d7327652 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:58.901030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 587/1186] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:50:58.906162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:58.906559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:58.906710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:58.923453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:50:58.924617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:50:58.988863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 600/1187] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:50:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:50:59.010969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086f77-e90c-4f59-af6c-b460d7327652 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:59.059723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:59.108947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c086f77-e90c-4f59-af6c-b460d7327652 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:59.141180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:59.141953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:59.142239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:59.142517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:59.152157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:50:59.159648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:50:59.161322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:50:59.162458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 588/1188] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:50:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:50:59.167275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:59.217720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:59.264335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2874d4-7d2c-4c49-9851-5a3b47c52acf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:59.296934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:59.297589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:59.297878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:59.298159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:59.298902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:50:59.299216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:59.300617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:59.308912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 601/1189] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:50:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:59.314491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:59.412605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde4963-41c6-4b00-b015-881f4ef9503b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:59.422356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:59.445746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:59.446608 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:59.446893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:59.447173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] PATH_INFO: `/v3/users/d091111ae2724958b0bfe4a7db800946` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:59.448383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d091111ae2724958b0bfe4a7db800946)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:59.449807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:59.451455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 589/1190] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:50:59 2022] GET /identity/v3/users/d091111ae2724958b0bfe4a7db800946 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:59.457006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:59.555018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62cf68f-7569-48be-8a3a-8e4e21bf97f9 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:59.567984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:50:59.588318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc68c0-695b-44e7-bd01-bd174d0c6220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:59.589164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc68c0-695b-44e7-bd01-bd174d0c6220 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:59.589450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc68c0-695b-44e7-bd01-bd174d0c6220 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:59.589727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc68c0-695b-44e7-bd01-bd174d0c6220 None admin] PATH_INFO: `/v3/projects/381a33da6ede46b5a43b730be06d5db0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:59.590898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc68c0-695b-44e7-bd01-bd174d0c6220 None admin] RBAC: Authorizing `identity:get_project(project_id=381a33da6ede46b5a43b730be06d5db0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:59.592656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc68c0-695b-44e7-bd01-bd174d0c6220 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:59.594808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 602/1191] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:50:59 2022] GET /identity/v3/projects/381a33da6ede46b5a43b730be06d5db0 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:50:59.601283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:50:59.696226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c6d05-385e-4436-a9a9-d70b4be198b9 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:50:59.728961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c92d55-8e27-4c05-9f2d-aa21b41f2f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:50:59.729830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92d55-8e27-4c05-9f2d-aa21b41f2f20 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:50:59.730118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92d55-8e27-4c05-9f2d-aa21b41f2f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:50:59.730400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c92d55-8e27-4c05-9f2d-aa21b41f2f20 None admin] PATH_INFO: `/v3/projects/381a33da6ede46b5a43b730be06d5db0/users/d091111ae2724958b0bfe4a7db800946/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:50:59.732374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c92d55-8e27-4c05-9f2d-aa21b41f2f20 None admin] RBAC: Authorizing `identity:create_grant(project_id=381a33da6ede46b5a43b730be06d5db0, user_id=d091111ae2724958b0bfe4a7db800946, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:50:59.733933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c92d55-8e27-4c05-9f2d-aa21b41f2f20 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:50:59.746213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 590/1192] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:50:59 2022] PUT /identity/v3/projects/381a33da6ede46b5a43b730be06d5db0/users/d091111ae2724958b0bfe4a7db800946/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:51:01.739532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e4dd-5acd-4f1c-9480-89994fe2e5af None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:01.740098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e4dd-5acd-4f1c-9480-89994fe2e5af None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:01.740098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5e4dd-5acd-4f1c-9480-89994fe2e5af None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:01.741474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 603/1193] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:51:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:51:01.746685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:01.747075 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:01.747273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:01.763464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:51:01.764659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:51:01.793503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:01.830721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 591/1194] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:51:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:51:01.855755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e4dd-5acd-4f1c-9480-89994fe2e5af None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:01.952206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5e4dd-5acd-4f1c-9480-89994fe2e5af None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:01.984917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:01.985693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:01.985978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:01.986252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:01.996499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:51:02.004183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:51:02.005317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:51:02.006232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 604/1195] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:51:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:51:02.011072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:02.106456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113feaf-fbea-46e0-b365-de0224747c07 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:02.139557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:02.140205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:02.140494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:02.140775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:02.141517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:51:02.141828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:02.143196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:02.150992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 592/1196] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:51:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:51:02.156726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:02.198535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:02.252547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4cc664-efc7-4e4e-9ca4-b26a4bf31cee None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:02.284201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:02.285046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:02.285328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:02.286116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] PATH_INFO: `/v3/users/d091111ae2724958b0bfe4a7db800946` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:02.287308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] RBAC: Authorizing `identity:get_user(user_id=d091111ae2724958b0bfe4a7db800946)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:02.288736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:02.290319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 605/1197] 10.222.0.45 () {56 vars in 1177 bytes} [Mon Jun 20 05:51:02 2022] GET /identity/v3/users/d091111ae2724958b0bfe4a7db800946 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:51:02.295852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:02.325758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:02.393498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831621f-987c-481d-a2f1-b68ebd4b8c4a None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:02.426057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:02.426710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:02.426987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:02.427277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:02.491761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:02.493197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:02.520982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:02.588215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 593/1198] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:51:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:51:02.594481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:02.690473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ee26a-4c15-4bf8-aead-3cfd10e49243 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:02.700799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:02.724734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fc46b7-6394-46ff-a321-b4b944b6ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:02.725573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc46b7-6394-46ff-a321-b4b944b6ab4f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:02.725861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc46b7-6394-46ff-a321-b4b944b6ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:02.726140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fc46b7-6394-46ff-a321-b4b944b6ab4f None admin] PATH_INFO: `/v3/projects/381a33da6ede46b5a43b730be06d5db0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:02.727335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc46b7-6394-46ff-a321-b4b944b6ab4f None admin] RBAC: Authorizing `identity:get_project(project_id=381a33da6ede46b5a43b730be06d5db0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:02.728758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fc46b7-6394-46ff-a321-b4b944b6ab4f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:02.730416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 606/1199] 10.222.0.45 () {56 vars in 1186 bytes} [Mon Jun 20 05:51:02 2022] GET /identity/v3/projects/381a33da6ede46b5a43b730be06d5db0 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:51:02.736317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:02.831094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd84ce-a272-4898-816d-85a096dca0c0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:02.859180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:02.865471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:02.866122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:02.866409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:02.866691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:02.867622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=381a33da6ede46b5a43b730be06d5db0, user.id=d091111ae2724958b0bfe4a7db800946) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:51:02.867943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:02.869361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d66c3c-9766-48e1-b73c-9a005c37e16d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:02.875087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 594/1200] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:51:02 2022] GET /identity/v3/role_assignments?user.id=d091111ae2724958b0bfe4a7db800946&scope.project.id=381a33da6ede46b5a43b730be06d5db0&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:51:12.720166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=11 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:12.872480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:57.287383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcdb35-16ae-4162-98df-f335792fa9d5 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:57.287976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcdb35-16ae-4162-98df-f335792fa9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:57.287976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bcdb35-16ae-4162-98df-f335792fa9d5 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:57.289313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 607/1201] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:51:57.298347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:57.298889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:57.298889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:57.318047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:51:57.319242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:51:57.399219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 595/1202] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:51:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:51:57.423718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcdb35-16ae-4162-98df-f335792fa9d5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:57.547923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bcdb35-16ae-4162-98df-f335792fa9d5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:57.593364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:57.594255 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:57.594546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:57.594829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:57.606106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:57.607840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:51:57.627365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:51:57.628715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:51:57.630021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:51:57.630021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:51:57.630021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:51:57.630021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:51:57.637197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:57.637373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 608/1203] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:51:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:51:57.754450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e12be-0bc3-4fbe-a285-8ed38f97f3f6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:57.789559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:57.798948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:57.799733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:57.800009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:57.800289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:57.801142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:51:57.801460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:57.802887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b39c9a-4353-4b34-b76d-d5beab76ea3f None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:57.823297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 596/1204] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:51:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:51:57.825813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:51:57.890107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:51:57.946921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5d5d13-5b49-4598-8e87-54bbbd67c888 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:51:57.994546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a844aaf3-5a06-4ea1-8c1f-27ca6cdf92ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:51:57.995450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844aaf3-5a06-4ea1-8c1f-27ca6cdf92ee None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:51:57.995740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844aaf3-5a06-4ea1-8c1f-27ca6cdf92ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:51:57.996116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a844aaf3-5a06-4ea1-8c1f-27ca6cdf92ee None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:51:57.996881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844aaf3-5a06-4ea1-8c1f-27ca6cdf92ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:51:57.999121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a844aaf3-5a06-4ea1-8c1f-27ca6cdf92ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:51:58.023292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 609/1205] 10.222.0.45 () {60 vars in 1132 bytes} [Mon Jun 20 05:51:57 2022] POST /identity/v3/users => generated 251 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:52:00.303927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f13f-9e7c-4ec5-a683-c18f77a7100b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:00.304474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f13f-9e7c-4ec5-a683-c18f77a7100b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:00.304474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d10f13f-9e7c-4ec5-a683-c18f77a7100b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:00.307361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 597/1206] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:00.311243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:00.311738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:00.311819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:00.329582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:00.330783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:00.411104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 610/1207] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:00.434316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f13f-9e7c-4ec5-a683-c18f77a7100b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:00.555652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d10f13f-9e7c-4ec5-a683-c18f77a7100b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:00.597264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:00.598070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:00.598355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:00.598627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:00.609680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:52:00.617475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:00.618772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:52:00.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:00.623766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 598/1208] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:52:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:00.624737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:00.670570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:00.742151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b204753-b457-4167-a745-8e5f54e7c900 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:00.790197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:00.791097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:00.791419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:00.791739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:00.792580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:00.792925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:00.794578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:00.807455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 611/1209] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:52:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:00.812969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:00.879428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:00.936020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787e4bc-a6e8-4f18-8ec7-575a47e03b8b None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:00.980285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:00.981383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:00.981624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:00.981904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:00.993747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:00.995226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:01.016810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:01.018037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:01.019582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:01.019582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:01.019582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:01.019582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:01.019582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 599/1210] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:01.024258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:01.146180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:01.149257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb1fdfb-ca6a-4b5a-8006-6b27b3aa1f05 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:01.190219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:01.190760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:01.191034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:01.191351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:01.192104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:01.192390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:01.193769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:01.208771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 612/1211] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:01.215684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:01.333059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a41ecc4-d25e-4ede-830e-5a815ea44b4d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:01.334714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:01.369975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:01.370781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:01.371088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:01.371401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:01.379508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:01.380937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:52:01.395519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:01.396613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:01.397619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 600/1212] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:01.402781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:01.513644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442856b7-4f38-4c53-baa8-9a4e3c4be52f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:01.550959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:01.551657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:01.551956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:01.552214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:01.553047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:01.553354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:01.554807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:01.579291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 613/1213] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Jun 20 05:52:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:01.582797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:01.714769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d898e3-a563-4e32-87be-28ab2ff78431 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:01.760062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:01.761113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:01.761458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:01.761736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:01.801687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:01.851691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:01.853256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:01.981235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 601/1214] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:52:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:52:01.987131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:02.008127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:02.097636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf39fd2-2ede-49c7-a6d8-2a903ebcfa3e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:02.136321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:02.137958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:02.137958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:02.137958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:02.148756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:02.150378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:02.169762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:02.170896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:02.171865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:02.171865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:02.171865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:02.171865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:02.171865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 614/1215] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:52:02.177421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:02.250048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:02.291350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1585c4-fd54-463c-8814-2d57ad6b50f1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:02.330044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:02.330829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:02.331177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:02.331588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:02.332515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:02.332914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:02.334438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:02.352623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 602/1216] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:52:02.354616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:02.431956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:02.469865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd3b7f-6015-4671-a807-d93a30d3ac52 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:02.511014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:02.511776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:02.512076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:02.512339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:02.519136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:02.520602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:52:02.534239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:02.535179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:02.536223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 615/1217] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:52:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:02.542631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:02.662412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b737c-e97c-47ff-8b08-d8a4caf49a00 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:02.687211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:02.700893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:02.701556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:02.701806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:02.702108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:02.702863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:02.703182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:02.704766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc57172-1ec8-49da-a80f-29e5b5877f77 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:02.717981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 603/1218] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:52:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:52:02.723796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:02.837913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97d852f-cd3a-4f42-bf41-59cb7f9e335b None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:02.872601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:02.880243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:02.880905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:02.881189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:02.881466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:02.882292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=e93c507849c04762bf311c0c3108bab3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:02.882602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:02.884069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c02e25f-462c-4594-b7d1-4e49bf197813 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:02.892193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 616/1219] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:52:02 2022] GET /identity/v3/role_assignments?user.id=e93c507849c04762bf311c0c3108bab3&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:05.131234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338dcd1-4afe-47a6-91ca-c1154c4e7e26 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:05.132275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338dcd1-4afe-47a6-91ca-c1154c4e7e26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:05.132275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338dcd1-4afe-47a6-91ca-c1154c4e7e26 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:05.133025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 604/1220] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:52:05.138551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:05.139084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:05.139479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:05.157185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:05.158467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:05.233376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 617/1221] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:05.262301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338dcd1-4afe-47a6-91ca-c1154c4e7e26 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:05.372901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338dcd1-4afe-47a6-91ca-c1154c4e7e26 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:05.410988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:05.411753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:05.412039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:05.412305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:05.423762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:52:05.431479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:05.433161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:05.434025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 605/1222] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:52:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:05.438912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:05.544900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119d5037-0cfa-4f8d-8ca3-9f011b9a5553 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:05.568586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:05.579682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:05.580343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:05.580624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:05.580897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:05.581631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] RBAC: Adding query filter params (name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:05.581939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:05.583338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:05.593472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 618/1223] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:52:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:05.599505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:05.646160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:05.709334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b01fc7-4951-4d72-ac8f-4a3f0ec5dc66 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:05.744716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:05.745634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:05.745994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:05.746364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:05.757374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:05.758931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:05.777884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:05.778979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:05.779958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:05.779958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:05.779958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:05.779958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:05.779958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 606/1224] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:52:05.785251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:05.892715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320eee0-a7e7-4226-96f3-11a0f71c1a7e None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:05.927370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:05.928041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:05.928326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:05.928677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:05.929429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:05.929771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:05.931294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:05.944120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 619/1225] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:05.950107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:06.049330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:06.056886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d0193-54b2-43ae-b125-4f1f85fd7010 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:06.093725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:06.095071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:06.095386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:06.095702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:06.104342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:06.105892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:52:06.121996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:06.123211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:06.124212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 607/1226] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 20 05:52:06.129429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:06.179717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:06.233428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe41f10-47b2-4695-8d2a-6b5f06788493 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:06.269191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:06.269874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:06.270159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:06.270432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:06.271275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:06.271592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:06.273025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:06.291750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 620/1227] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Jun 20 05:52:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:06.298351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:06.406818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895dc02-7814-4fab-86d5-eb11fdd293a8 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:06.442742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:06.443607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:06.443907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:06.444172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:06.455116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:06.456577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:06.474295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:06.475580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:06.476508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:06.476508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:06.476508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:06.476508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:06.476508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 608/1228] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:06.481825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:06.582906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e24b8c-dbc0-4925-9760-3fbbe3817550 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:06.585569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:06.623464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:06.624257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:06.624544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:06.625075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:06.626488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:06.626790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:06.628311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:06.641210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 621/1229] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:06.650800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:06.660796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:06.772578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814f3ec-9439-4ade-a89e-e9de2e877fae None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:06.809046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:06.809912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:06.810200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:06.810476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:06.821476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:06.823060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:52:06.841005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:06.842111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:06.843077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 609/1230] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:52:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:06.848102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:06.955847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ff3c1-5af8-417e-a7cc-011c815ced0c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:06.992155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:06.992821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:06.993152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:06.993474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:06.994304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:06.994620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:06.996206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7bdfd6-bc85-4939-832a-32d6f3a4a0be None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:07.008550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 622/1231] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:52:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:07.014962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:07.085547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:07.122652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9734b-131f-4130-849d-03ca6d8bda09 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:07.157427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee046835-d783-4388-958c-a42843a99114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:07.158315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046835-d783-4388-958c-a42843a99114 None admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:07.158601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046835-d783-4388-958c-a42843a99114 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:07.158896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee046835-d783-4388-958c-a42843a99114 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/e93c507849c04762bf311c0c3108bab3/roles/566975a26650405d88a512eb4c0913e5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:07.174258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046835-d783-4388-958c-a42843a99114 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=e93c507849c04762bf311c0c3108bab3, role_id=566975a26650405d88a512eb4c0913e5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:07.175733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee046835-d783-4388-958c-a42843a99114 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:07.190087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 610/1232] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:52:07 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/e93c507849c04762bf311c0c3108bab3/roles/566975a26650405d88a512eb4c0913e5 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:52:09.282714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68e17-254b-4d52-8702-47a5420d722e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:09.283434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68e17-254b-4d52-8702-47a5420d722e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:09.283434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e68e17-254b-4d52-8702-47a5420d722e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:09.284771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 623/1233] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:09.290322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:09.290847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:09.290952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:09.308491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:09.309796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:09.381198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 611/1234] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:09.408755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e68e17-254b-4d52-8702-47a5420d722e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:09.439961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:09.527564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e68e17-254b-4d52-8702-47a5420d722e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:09.567641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:09.568383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:09.568672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:09.568951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:09.582735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:52:09.591937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:09.593195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 20 05:52:09.594340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:09.601368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 624/1235] 10.222.0.45 () {56 vars in 1102 bytes} [Mon Jun 20 05:52:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:09.602436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:09.657543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:09.723677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5264a-c198-461b-ad86-cd9a101f1194 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:09.760305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:09.760983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:09.761271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:09.761559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:09.762296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] RBAC: Adding query filter params (name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:09.762604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:09.763996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:09.773342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 612/1236] 10.222.0.45 () {56 vars in 1103 bytes} [Mon Jun 20 05:52:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:52:09.779790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:09.881524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:09.898670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d673af24-8610-49a4-93f3-323df844f9b5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:09.936605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:09.937504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:09.938429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:09.938719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:09.952834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:09.954969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:09.985573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:09.986679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:09.992346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:09.992346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:09.992346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:09.992346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:09.992346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 625/1237] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:09.993757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:10.105536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6efbaf-116e-46cd-aa53-0510aa801436 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:10.115373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:10.142962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:10.143666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:10.143957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:10.144239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:10.144962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:10.145271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:10.146646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:10.158540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 613/1238] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:10.164214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:10.279685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ab72aa-8fff-4da1-9cba-430e3ef711b8 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:10.317221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:10.318024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:10.318309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:10.318570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:10.321529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:10.327544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:10.329233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:52:10.344619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:10.345892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:10.346982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 626/1239] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:10.352382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:10.460050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ccdf4-6bec-4e51-8ac8-9dd4243a0e64 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:10.496071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:10.496735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:10.497026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:10.497307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:10.498149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:10.498465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:10.499926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:10.517351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 614/1240] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Jun 20 05:52:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:10.523102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:10.637168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f80e4a-24b4-45b6-9732-09226a407870 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:10.675014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:10.675695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:10.675981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:10.676258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:10.741981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:10.748945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:10.750594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:10.860093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 627/1241] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:52:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:52:10.866272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:10.896970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:10.979147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a48ecc-fcc4-475a-9a9e-58485bd80df0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:11.018567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:11.019286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:11.019568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:11.019915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:11.031751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:11.033346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:11.052579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:11.053582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:11.054502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:11.054502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:11.054502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:11.054502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:11.054502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 615/1242] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:11.060141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:11.165261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:11.171985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ef192-8cd7-4ee1-878d-559db2d98b48 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:11.211595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:11.212258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:11.212543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:11.212818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:11.213534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:11.213846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:11.215214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:11.228431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 628/1243] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:11.234150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:11.319632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:11.348318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f6da5-bc02-42a1-921f-855a0743d134 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:11.384583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:11.385444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:11.385785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:11.386131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:11.393962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:11.395583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:52:11.410218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:11.411370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:11.412696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 616/1244] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:52:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:11.417359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:11.522671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9234ded1-91f0-4f15-95cd-a76af6c9e2ac None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:11.561106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:11.562163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:11.562517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:11.562801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:11.563665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:11.563986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:11.565535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc93da1-7e87-4bc6-8e55-37c8567d0241 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:11.578963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 629/1245] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:52:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:11.585608 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:11.701017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff1ba33-c27e-4dcf-8eb1-5e63c0a76009 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:11.740032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:11.740709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:11.741210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:11.741351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:11.742349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] RBAC: Adding query filter params (role.id=566975a26650405d88a512eb4c0913e5, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=e93c507849c04762bf311c0c3108bab3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:11.742716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:11.744352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b70382-9dc2-4f83-a556-caede8a35be6 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:11.750534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 617/1246] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:52:11 2022] GET /identity/v3/role_assignments?user.id=e93c507849c04762bf311c0c3108bab3&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=566975a26650405d88a512eb4c0913e5 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:52:14.039621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1974-2694-49d8-8f7e-537ae822b27b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:14.040167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1974-2694-49d8-8f7e-537ae822b27b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:14.040167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1974-2694-49d8-8f7e-537ae822b27b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:14.041549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 630/1247] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:52:14.046959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:14.047401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:14.047563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:14.053170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:14.066225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:14.067552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:14.142676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 618/1248] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:14.166091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1974-2694-49d8-8f7e-537ae822b27b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:14.186078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:14.273070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1974-2694-49d8-8f7e-537ae822b27b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:14.309047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:14.309830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:14.310158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:14.310479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:14.322084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:52:14.330193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:14.331161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:14.332223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 631/1249] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:52:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:14.337718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:14.443702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5135fc6-84f0-4c57-ae42-5da81197b2c0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:14.476339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:14.481740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:14.482523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:14.482876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:14.483402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:14.484691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:14.485071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:14.486595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:14.495773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 619/1250] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:52:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:14.502451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:14.588814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:14.618121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435c5bf-d740-4c7d-97d5-8804245deb55 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:14.657979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:14.658881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:14.659217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:14.659571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:14.674321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:14.674321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:14.693654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:14.694722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:14.695686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:14.695686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:14.695686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:14.695686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:14.695686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 632/1251] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:14.701165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:14.820482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2421a50-ef43-467b-b478-39184b15f7c4 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:14.862491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:14.863379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:14.863673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:14.864028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:14.864945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:14.865419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:14.867605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:14.883129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 620/1252] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:14.888824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:14.997497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ba9a8-f61a-4720-b6e8-6a5d6be75130 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:15.025577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:15.035751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:15.036563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:15.036936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:15.037262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:15.045462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:15.047759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:52:15.063676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:15.065043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:15.066058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 633/1253] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:15.071473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:15.128121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:15.173876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81308756-b3b9-48d0-a5f0-29c437724ff3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:15.212280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:15.212955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:15.213242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:15.213516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:15.214348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:15.214657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:15.216466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:15.234300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 621/1254] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Jun 20 05:52:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:52:15.240535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:15.356936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3b12f-dc21-47d2-8966-e7560ed66914 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:15.391828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:15.392598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:15.392935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:15.393239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:15.453499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:15.466912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:15.468417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:15.568860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 634/1255] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:52:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:52:15.574743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:15.685809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dcf65d-3271-4fbf-8446-8b25ff164f39 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:15.701171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:15.725322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:15.726224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:15.726509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:15.726789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:15.737292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:15.738732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:15.756424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:15.757518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:15.758736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:15.758736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:15.758736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:15.758736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:15.758736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 622/1256] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:15.763415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:15.855794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:15.868536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1245f486-2d31-4df6-9405-3b47c927e8a3 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:15.901826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:15.902480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:15.904560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:15.904833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:15.905563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:15.906635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:15.907837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:15.919175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 635/1257] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 05:52:15.925162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:16.033727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada4020-e2b8-4ab5-a7d8-4309b99c0487 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:16.074140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:16.074941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:16.075231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:16.075536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:16.083295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:16.084591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:52:16.103127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:16.104301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:16.105415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 623/1258] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:52:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:16.110808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:16.216544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578431e5-9eab-4e3b-9257-7f8f9648e774 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:16.251633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:16.252284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:16.252615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:16.252897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:16.253750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:16.254105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:16.255631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5477f043-a815-49fc-a895-bdf862b70f9c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:16.268797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 636/1259] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:52:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:16.274952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:16.302237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:16.398845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c1fb0-76ff-4a73-a765-8b7c87675b4d None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:16.433822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:16.434602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:16.434893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:16.435175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:16.436071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=e93c507849c04762bf311c0c3108bab3) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:16.436391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:16.438383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cb9aba-22ac-4365-bafa-5b52358a7740 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:16.444861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 624/1260] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:52:16 2022] GET /identity/v3/role_assignments?user.id=e93c507849c04762bf311c0c3108bab3&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:18.431044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593fa76-e816-47b2-9b57-8fbbcf8f30a0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:18.431614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593fa76-e816-47b2-9b57-8fbbcf8f30a0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:18.431614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5593fa76-e816-47b2-9b57-8fbbcf8f30a0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:18.432979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 637/1261] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:18.438096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:18.438462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:18.438664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:18.455511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:18.456699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:18.521384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 625/1262] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:18.543744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593fa76-e816-47b2-9b57-8fbbcf8f30a0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:18.554570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:18.647494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5593fa76-e816-47b2-9b57-8fbbcf8f30a0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:18.684731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:18.685456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:18.685743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:18.686024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:18.696307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:52:18.704512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:18.705597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:18.706643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 638/1263] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:52:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:18.711799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:18.729510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:18.806351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320e6ff-c0a7-43eb-b7bb-ef00c4136a3d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:18.840259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:18.841180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:18.841466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:18.841741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:18.842476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:18.842781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:18.844189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:18.851917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 626/1264] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:52:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:18.857512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:18.922207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:18.955816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7844fb-3107-45ca-976e-a0fa2dfb582f None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:18.988181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:18.988948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:18.989233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:18.989508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:18.999718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.001149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:19.018160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:19.019202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:19.020339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:19.020339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:19.020339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:19.020339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:19.020339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 639/1265] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:19.025066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:19.102547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:19.121705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5730ff7-4f20-47c2-90f8-07dd8c6ad15e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:19.153352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:19.153977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:19.154261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:19.154535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:19.155242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:19.155570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.157527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.169865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 627/1266] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:19.175523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:19.270783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e47ba-c1ad-4ba9-ab90-786e68135fc5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:19.289719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:19.303334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:19.304118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:19.304402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:19.304678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:19.311830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.313776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:52:19.328448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:19.329549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:19.330682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 640/1267] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:19.335405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:19.429153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6356946d-7d41-41e1-b8d4-985a51cf3d40 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:19.461867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:19.462520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:19.462806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:19.463082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:19.463934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:19.464242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.465908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.474149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:19.482607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 628/1268] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:19.488088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:19.584385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbf8a2e-398a-4dd1-9bb2-b6520fbd2014 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:19.618384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:19.619323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:19.619616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:19.619998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:19.631752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.633423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:19.653005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:19.654084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:19.655199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:19.655199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:19.655199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:19.655199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:19.655199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 641/1269] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:19.659921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:19.757175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8d4de0-4fbd-45a9-bb63-a379658e7917 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:19.789161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:19.789821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:19.790109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:19.790388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:19.791111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:19.791451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.792801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.804145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 629/1270] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:19.809785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:19.841513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:19.907027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e3c69-f6a2-4f74-8dc0-c73cfef0252c None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:19.939342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:19.940124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:19.940408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:19.940683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:19.947235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:19.948714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:52:19.962675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:19.963806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:19.964925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 642/1271] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:19.969914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:20.026606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:20.067802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba12c4d1-41d8-45d6-b269-2ec3e2646c2e None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:20.100419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:20.101007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:20.101293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:20.101572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:20.102355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:20.102672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:20.104120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d9009e-696a-46c6-8a2c-f735ed00e85a None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:20.116536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 630/1272] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:52:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:20.126248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:20.212420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:20.221779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bc5d1-03a7-4297-b879-2fb5346b5c25 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:20.254778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2abed4-0426-44fb-8521-c79c350ddb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:20.255679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2abed4-0426-44fb-8521-c79c350ddb81 None admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:20.255962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2abed4-0426-44fb-8521-c79c350ddb81 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:20.256234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2abed4-0426-44fb-8521-c79c350ddb81 None admin] PATH_INFO: `/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/e93c507849c04762bf311c0c3108bab3/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:20.258131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2abed4-0426-44fb-8521-c79c350ddb81 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a62eb9ba0184f96ac14a0536d5b646e, user_id=e93c507849c04762bf311c0c3108bab3, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:20.259578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2abed4-0426-44fb-8521-c79c350ddb81 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:20.271417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 643/1273] 10.222.0.45 () {58 vars in 1439 bytes} [Mon Jun 20 05:52:20 2022] PUT /identity/v3/projects/1a62eb9ba0184f96ac14a0536d5b646e/users/e93c507849c04762bf311c0c3108bab3/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:52:22.286534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74fb7-c67e-482b-8c42-b71aba39f20d None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:22.287089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74fb7-c67e-482b-8c42-b71aba39f20d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:22.287089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b74fb7-c67e-482b-8c42-b71aba39f20d None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:22.288501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 631/1274] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:22.293602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:22.294031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:22.294170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:22.310249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:22.311467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:22.376640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 644/1275] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:22.400174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74fb7-c67e-482b-8c42-b71aba39f20d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:22.497693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b74fb7-c67e-482b-8c42-b71aba39f20d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:22.514490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:22.532180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:22.532972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:22.533261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:22.533540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:22.543329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 20 05:52:22.550806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:22.551974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:22.552840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 632/1276] 10.222.0.45 () {56 vars in 1096 bytes} [Mon Jun 20 05:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:22.557819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:22.646865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:22.665640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdf077-427f-4326-a9bb-42cb94f6616e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:22.698817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:22.699514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:22.699813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:22.700097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:22.700838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:22.701150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:22.702542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:22.710803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 645/1277] 10.222.0.45 () {56 vars in 1099 bytes} [Mon Jun 20 05:52:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:22.716651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:22.813677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7d147-a8c6-45c2-a9a7-2854363ba2c1 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:22.846759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:22.847585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:22.847874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:22.848151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:22.858324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:22.860300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:22.877308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:22.878331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:22.879613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:22.879613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:22.879613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:22.879613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:22.879613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 633/1278] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:22.884119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:22.982606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21029cbc-dac3-422b-8bb9-bab9562e5f71 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:23.015473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:23.016124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:23.016411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:23.016691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:23.017398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:23.017706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:23.019066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:23.030132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 646/1279] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:23.036225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:23.066311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:23.131975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38025947-08ba-400e-97c3-90f93a9b4abe None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:23.164220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:23.165014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:23.165302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:23.165582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:23.173330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:23.174754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 20 05:52:23.188682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:23.189803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:23.190774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 634/1280] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:23.195497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:23.199136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:23.291211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac1f7b6-6d8c-4f26-b803-98e93d27b8c5 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:23.323094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:23.323724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:23.324006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:23.324283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:23.325099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:23.325410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:23.326833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:23.343943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 647/1281] 10.222.0.45 () {56 vars in 1143 bytes} [Mon Jun 20 05:52:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:23.349489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:23.437274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:23.447090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84696cab-ee95-48f9-919b-14eda568fae2 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:23.478975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:23.479635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:23.479919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:23.480196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:23.543912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:23.545344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:23.634277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:23.644989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 635/1282] 10.222.0.45 () {58 vars in 1282 bytes} [Mon Jun 20 05:52:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 20 05:52:23.650873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:23.709829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:23.746914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16edb360-f95a-44ed-b968-7304a0ce412d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:23.779541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:23.780319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:23.781128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:23.781407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:23.790974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:23.792440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 20 05:52:23.810463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:23.811524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:23.812732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 20 05:52:23.812732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 20 05:52:23.812732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 20 05:52:23.812732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:23.812732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 648/1283] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:23.817274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:23.911505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9620aa-fbcf-4e70-b0ab-b950f02d24a0 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:23.943557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:23.944202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:23.944489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:23.944762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:23.945469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:23.945770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:23.947107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:23.957851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 636/1284] 10.222.0.45 () {56 vars in 1109 bytes} [Mon Jun 20 05:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:23.963588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:24.061045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f322436-cd65-42ae-a15c-b0bd22db703d None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:24.074296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:24.094317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:24.095091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:24.095406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:24.095687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:24.102068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:24.103536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 20 05:52:24.116972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:24.118029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 05:52:24.119182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 649/1285] 10.222.0.45 () {56 vars in 1111 bytes} [Mon Jun 20 05:52:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:24.123905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:24.173188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:24.220589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b9e66-ed2a-4851-af26-be8ed5173347 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:24.252272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:24.252917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:24.253199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:24.253472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:24.254242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:24.254550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:24.255969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91586d40-e84d-4dd4-a2ba-07e9016018a5 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:24.267822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 637/1286] 10.222.0.45 () {56 vars in 1148 bytes} [Mon Jun 20 05:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:24.273926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:24.369476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c680006-89f3-4a3c-b50f-40ab67e73d52 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:24.401173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:24.401804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:24.402084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:24.402359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:24.403170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] RBAC: Adding query filter params (role.id=f7649bd8aa81487b8ca1fbc0e5fcba91, scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e, user.id=e93c507849c04762bf311c0c3108bab3) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:24.403507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:24.404922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d57f3c2-f058-4f4e-a330-758d2c2b30ee None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:24.410445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 650/1287] 10.222.0.45 () {56 vars in 1374 bytes} [Mon Jun 20 05:52:24 2022] GET /identity/v3/role_assignments?user.id=e93c507849c04762bf311c0c3108bab3&scope.project.id=1a62eb9ba0184f96ac14a0536d5b646e&role.id=f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:26.422497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040feeeb-2edc-4761-b40b-1eaacb288c40 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:26.423021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040feeeb-2edc-4761-b40b-1eaacb288c40 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:26.423021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040feeeb-2edc-4761-b40b-1eaacb288c40 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:26.424429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 638/1288] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:26.429553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:26.429953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:26.430107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:26.446944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:26.448139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:26.476266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:26.512633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 651/1289] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:26.534804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040feeeb-2edc-4761-b40b-1eaacb288c40 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:26.634336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040feeeb-2edc-4761-b40b-1eaacb288c40 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:26.646854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:26.674379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:26.675154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:26.675543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:26.675844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:26.676502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:26.677862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:26.690172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:26.691360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:26.692361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 639/1290] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:52:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:26.697285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:26.790964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c7900-7143-45e0-a0e9-693250a960a0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:26.824744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:26.825398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:26.825681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:26.825958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:26.826655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:26.826963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:26.828391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73436561-2e15-447e-b418-f22267504dd2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:26.833073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 652/1291] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Jun 20 05:52:26 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:26.838615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:26.933789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b9b71-9a38-476d-8f36-795f6f1f7b99 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:26.964816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:26.965718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:26.966000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:26.966272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] PATH_INFO: `/v3/services` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:26.966960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] RBAC: Adding query filter params (type=placement) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:26.967290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:26.969138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42109aaa-9e52-4258-8256-a3669294d66e None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:26.973352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 640/1292] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Jun 20 05:52:26 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:28.945019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81068b22-402a-4a8d-89d5-4d18865f4665 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:28.945524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81068b22-402a-4a8d-89d5-4d18865f4665 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:28.945668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81068b22-402a-4a8d-89d5-4d18865f4665 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:28.947032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 653/1293] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:28.952196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5f43-a656-4162-b56e-9bb32cd039d3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:28.952616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5f43-a656-4162-b56e-9bb32cd039d3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:28.952766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaf5f43-a656-4162-b56e-9bb32cd039d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:28.969362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-aaaf5f43-a656-4162-b56e-9bb32cd039d3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:28.970550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aaaf5f43-a656-4162-b56e-9bb32cd039d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:28.999636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:29.035561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 641/1294] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:29.058559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81068b22-402a-4a8d-89d5-4d18865f4665 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:29.088176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:29.156751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81068b22-402a-4a8d-89d5-4d18865f4665 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:29.189430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f65ad8-8506-4085-9591-e295f75b0255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:29.190067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f65ad8-8506-4085-9591-e295f75b0255 None admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:29.190348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f65ad8-8506-4085-9591-e295f75b0255 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:29.190623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f65ad8-8506-4085-9591-e295f75b0255 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:29.191364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f65ad8-8506-4085-9591-e295f75b0255 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:29.192717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f65ad8-8506-4085-9591-e295f75b0255 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:29.205210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 654/1295] 10.222.0.45 () {60 vars in 1141 bytes} [Mon Jun 20 05:52:29 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:52:31.199164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe5519-fa41-49fb-9d90-e7d40c270882 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:31.199822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe5519-fa41-49fb-9d90-e7d40c270882 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:31.199822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe5519-fa41-49fb-9d90-e7d40c270882 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:31.201109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 642/1296] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:31.206644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:31.207034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:31.207321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:31.224444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:31.225610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:31.295513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 655/1297] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:31.318215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe5519-fa41-49fb-9d90-e7d40c270882 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:31.415538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe5519-fa41-49fb-9d90-e7d40c270882 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:31.447981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:31.448760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:31.449042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:31.449367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:31.449964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:31.451389 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:31.462858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:31.464222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:31.465629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 643/1298] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:52:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:31.470764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:31.488791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:31.565508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4d745d-3146-4639-8c0f-b34f83e7fe92 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:31.597716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:31.598358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:31.598638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:31.598913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:31.599633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:31.599937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:31.601292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:31.606214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 656/1299] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Jun 20 05:52:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:31.612597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:31.648410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:31.721804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5eb86b-6b59-4637-8552-57a6641e21f3 None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:31.753276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:31.753934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:31.754222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:31.754504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:31.755311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] RBAC: Adding query filter params (interface=public, service_id=880a30b9f72e48b2ba0838f43f068015, region_id=RegionOne) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:31.755631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:31.756988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b25c2-f6be-46dc-86d5-a60ebc98819b None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:31.761856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 644/1300] 10.222.0.45 () {56 vars in 1251 bytes} [Mon Jun 20 05:52:31 2022] GET /identity/v3/endpoints?service_id=880a30b9f72e48b2ba0838f43f068015&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:31.768034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:31.845971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:31.864189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17f6a83-9e8f-4a87-9819-5d6040c9e5b7 None admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:31.896519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a8542-1db0-4b03-a8bd-b9dfe69857f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:31.897167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a8542-1db0-4b03-a8bd-b9dfe69857f2 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:31.897448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a8542-1db0-4b03-a8bd-b9dfe69857f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:31.897724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a8542-1db0-4b03-a8bd-b9dfe69857f2 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:31.898392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a8542-1db0-4b03-a8bd-b9dfe69857f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:31.899782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5a8542-1db0-4b03-a8bd-b9dfe69857f2 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:31.906349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 657/1301] 10.222.0.45 () {56 vars in 1087 bytes} [Mon Jun 20 05:52:31 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:52:33.896065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90453c-2b96-43e4-864a-8841e475e0c6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:33.896584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90453c-2b96-43e4-864a-8841e475e0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:33.896684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e90453c-2b96-43e4-864a-8841e475e0c6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:33.898001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 645/1302] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:52:33.903205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:33.903628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:33.903787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:33.920072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:52:33.921769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:52:33.986123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 658/1303] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:52:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 20 05:52:34.009101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90453c-2b96-43e4-864a-8841e475e0c6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:34.101506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:34.107502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e90453c-2b96-43e4-864a-8841e475e0c6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:34.140686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:34.141477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:34.141767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:34.142048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:34.142697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:34.144091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 05:52:34.155763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 05:52:34.156909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application with Lock( Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self._enter() Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application return self.creator() Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application created_value = creator( Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: ERROR keystone.server.flask.application Jun 20 05:52:34.157949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 646/1304] 10.222.0.45 () {56 vars in 1117 bytes} [Mon Jun 20 05:52:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 20 05:52:34.162771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:34.239730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:34.262589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf55bb2-81b9-43b7-8581-0ac4ae78a633 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:34.295090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:34.296313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:34.296597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:34.296878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] PATH_INFO: `/v3/services` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:34.297578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:52:34.297889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:34.299287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f931f2-28bc-4c0f-a962-5fd4f836a400 None admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:34.304277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 659/1305] 10.222.0.45 () {56 vars in 1116 bytes} [Mon Jun 20 05:52:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:52:34.309967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:52:34.405640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe90d7-2ea1-472d-8002-054de40c51bf None admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:52:34.437541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f649ae55-5cef-49d4-9b5b-6c4c22c1bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:52:34.438178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ae55-5cef-49d4-9b5b-6c4c22c1bfc9 None admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:52:34.438466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ae55-5cef-49d4-9b5b-6c4c22c1bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:52:34.438744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f649ae55-5cef-49d4-9b5b-6c4c22c1bfc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:52:34.439496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649ae55-5cef-49d4-9b5b-6c4c22c1bfc9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:52:34.440847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f649ae55-5cef-49d4-9b5b-6c4c22c1bfc9 None admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:52:34.458410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 647/1306] 10.222.0.45 () {60 vars in 1144 bytes} [Mon Jun 20 05:52:34 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:52:44.301816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=30 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:52:44.454572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:36.183241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bee7ad-1205-4362-9928-6ddcefa93cc4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:36.183856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bee7ad-1205-4362-9928-6ddcefa93cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:36.183856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bee7ad-1205-4362-9928-6ddcefa93cc4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:36.185224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 660/1307] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:36.191170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f691c1-980b-4f5d-b0ad-e0bfdfd37a90 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:36.191702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f691c1-980b-4f5d-b0ad-e0bfdfd37a90 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:36.192273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f691c1-980b-4f5d-b0ad-e0bfdfd37a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:36.282802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-08f691c1-980b-4f5d-b0ad-e0bfdfd37a90 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:36.284466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08f691c1-980b-4f5d-b0ad-e0bfdfd37a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:36.403509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 648/1308] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:36.462908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace20e-c38b-4b2e-988c-f2004e876d66 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:36.463270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace20e-c38b-4b2e-988c-f2004e876d66 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:36.463983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ace20e-c38b-4b2e-988c-f2004e876d66 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:36.465946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 661/1309] 10.222.0.45 () {54 vars in 964 bytes} [Mon Jun 20 05:53:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:36.481038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac63b8-32ca-4086-8eed-1bbdd9c8c089 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:36.481243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac63b8-32ca-4086-8eed-1bbdd9c8c089 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:36.481523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac63b8-32ca-4086-8eed-1bbdd9c8c089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:36.632544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fbac63b8-32ca-4086-8eed-1bbdd9c8c089 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:36.634060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fbac63b8-32ca-4086-8eed-1bbdd9c8c089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:36.659310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:36.776590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 649/1310] 10.222.0.45 () {58 vars in 1061 bytes} [Mon Jun 20 05:53:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:36.777653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace20e-c38b-4b2e-988c-f2004e876d66 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:36.779562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-35ace20e-c38b-4b2e-988c-f2004e876d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:36.927300 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ace20e-c38b-4b2e-988c-f2004e876d66 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:37.022274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:37.023453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:37.024053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:37.024549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:37.026702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:37.096607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:37.164101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:37.166045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1146ec63-cc35-4730-a0dc-59415bca57bd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:37.202629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 662/1311] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 05:53:39.541930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df107c7c-7169-4946-a8de-e588b5032f62 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:39.542467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df107c7c-7169-4946-a8de-e588b5032f62 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:39.542467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df107c7c-7169-4946-a8de-e588b5032f62 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:39.543884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 650/1312] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:39.548927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05b6c-11c1-4fae-be7c-957994583fd9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:39.549348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05b6c-11c1-4fae-be7c-957994583fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:39.549494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f05b6c-11c1-4fae-be7c-957994583fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:39.620772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-83f05b6c-11c1-4fae-be7c-957994583fd9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:39.622001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-83f05b6c-11c1-4fae-be7c-957994583fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:39.717544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 663/1313] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:39.756234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df107c7c-7169-4946-a8de-e588b5032f62 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:39.757768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-df107c7c-7169-4946-a8de-e588b5032f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:39.861014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df107c7c-7169-4946-a8de-e588b5032f62 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:39.936296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:39.939240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:39.940146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:39.940590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:39.940986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:39.942957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:40.057738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:40.059157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935923a-68b9-450f-afe9-036f6ef5b885 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:40.077368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 651/1314] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:42.391731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd8c6-c4e2-4e89-aab9-b10375c08545 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:42.392237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd8c6-c4e2-4e89-aab9-b10375c08545 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:42.392348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd8c6-c4e2-4e89-aab9-b10375c08545 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:42.393702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 664/1315] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:42.398657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439779-10b6-4d08-85eb-4c0a201bcddf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:42.399096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439779-10b6-4d08-85eb-4c0a201bcddf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:42.399222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6439779-10b6-4d08-85eb-4c0a201bcddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:42.461201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b6439779-10b6-4d08-85eb-4c0a201bcddf None None] MFA Rules not processed for user `31594a42591343ab884a5efd018433c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:42.462411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b6439779-10b6-4d08-85eb-4c0a201bcddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:42.512268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:42.555441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 652/1316] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:42 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:42.609656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44660c30-58fb-4aa4-ad74-ecbb55329191 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:42.609949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44660c30-58fb-4aa4-ad74-ecbb55329191 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:42.610229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44660c30-58fb-4aa4-ad74-ecbb55329191 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:42.611729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 665/1317] 10.222.0.45 () {54 vars in 964 bytes} [Mon Jun 20 05:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:42.618058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03437-ffab-44d7-bcfb-28ae584b14c5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:42.618418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03437-ffab-44d7-bcfb-28ae584b14c5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:42.618739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03437-ffab-44d7-bcfb-28ae584b14c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:42.691156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-23b03437-ffab-44d7-bcfb-28ae584b14c5 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:42.692380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-23b03437-ffab-44d7-bcfb-28ae584b14c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:42.766123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 653/1318] 10.222.0.45 () {58 vars in 1061 bytes} [Mon Jun 20 05:53:42 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:42.776239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44660c30-58fb-4aa4-ad74-ecbb55329191 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:42.777794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-44660c30-58fb-4aa4-ad74-ecbb55329191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:42.787567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:42.881017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44660c30-58fb-4aa4-ad74-ecbb55329191 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:42.951695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:42.952342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:42.952631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:42.952913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:42.954561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:43.015210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:43.060444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:43.061865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e1939-5492-4f43-afbb-390121aa2ff6 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:43.079579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 666/1319] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:45.989763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242e40b-1076-405f-a31e-06600ff0a75c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:45.990289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242e40b-1076-405f-a31e-06600ff0a75c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:45.990386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242e40b-1076-405f-a31e-06600ff0a75c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:45.991760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 654/1320] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:45.997360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b3fb6-55fc-45bb-a1d2-67e4ec853f7a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:45.997798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b3fb6-55fc-45bb-a1d2-67e4ec853f7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:45.997921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80b3fb6-55fc-45bb-a1d2-67e4ec853f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:46.060874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c80b3fb6-55fc-45bb-a1d2-67e4ec853f7a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:46.062071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c80b3fb6-55fc-45bb-a1d2-67e4ec853f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:46.154543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 667/1321] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:46.193907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242e40b-1076-405f-a31e-06600ff0a75c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:46.195486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f242e40b-1076-405f-a31e-06600ff0a75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:46.243963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:46.297178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f242e40b-1076-405f-a31e-06600ff0a75c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:46.372716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:46.373360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:46.373643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:46.373919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:46.375580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:46.467122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:46.480459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:46.481879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d240bec-d927-4d7e-906b-31545e4c400a service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:46.499190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 655/1322] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:46 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:49.187531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987bfe9-6077-4424-984b-ca33d355bae7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:49.188087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987bfe9-6077-4424-984b-ca33d355bae7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:49.188087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987bfe9-6077-4424-984b-ca33d355bae7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:49.189467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 668/1323] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:49.194759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a352-4341-4bd7-abcc-f3c18d20a57a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:49.195195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a352-4341-4bd7-abcc-f3c18d20a57a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:49.195367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf4a352-4341-4bd7-abcc-f3c18d20a57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:49.259307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4cf4a352-4341-4bd7-abcc-f3c18d20a57a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:49.261026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4cf4a352-4341-4bd7-abcc-f3c18d20a57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:49.360428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 656/1324] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:49.397747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987bfe9-6077-4424-984b-ca33d355bae7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:49.399346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7987bfe9-6077-4424-984b-ca33d355bae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:49.419182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:49.508967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987bfe9-6077-4424-984b-ca33d355bae7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:49.583414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:49.584072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:49.584354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:49.584632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:49.586278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:49.661991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:49.700798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:49.702256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312442d-5805-4984-9b83-5f656d5e60fb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:49.720789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 669/1325] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:51.818321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ff19-2c5d-443b-84f2-18f413aa5d6b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:51.818824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ff19-2c5d-443b-84f2-18f413aa5d6b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:51.818824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a491ff19-2c5d-443b-84f2-18f413aa5d6b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:51.820198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 657/1326] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:51.826107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb4ae-2c1a-4674-a8ee-c1984279ea45 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:51.826517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb4ae-2c1a-4674-a8ee-c1984279ea45 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:51.826671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bdb4ae-2c1a-4674-a8ee-c1984279ea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:51.890748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-56bdb4ae-2c1a-4674-a8ee-c1984279ea45 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:51.892025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-56bdb4ae-2c1a-4674-a8ee-c1984279ea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:51.989356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 670/1327] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:52.025206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ff19-2c5d-443b-84f2-18f413aa5d6b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:52.026765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a491ff19-2c5d-443b-84f2-18f413aa5d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:52.086825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:52.132042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a491ff19-2c5d-443b-84f2-18f413aa5d6b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:52.209434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:52.210091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:52.210378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:52.210659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:52.212355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:52.322610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:52.325403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:52.327641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426ccf8-0b7a-4bfc-8a4c-007aa4003fb8 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:52.346867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 658/1328] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:54.436246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546a419-3b6d-43da-8f1b-8cefa8a5b8b4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:54.436770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546a419-3b6d-43da-8f1b-8cefa8a5b8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:54.436892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c546a419-3b6d-43da-8f1b-8cefa8a5b8b4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:54.438172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 671/1329] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:54.443303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bf0f4-f710-4271-bf98-54af6ed9222e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:54.443689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bf0f4-f710-4271-bf98-54af6ed9222e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:54.443994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2bf0f4-f710-4271-bf98-54af6ed9222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:54.511359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6b2bf0f4-f710-4271-bf98-54af6ed9222e None None] MFA Rules not processed for user `31594a42591343ab884a5efd018433c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:54.512584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6b2bf0f4-f710-4271-bf98-54af6ed9222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:54.613066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 659/1330] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:54 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:54.649982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546a419-3b6d-43da-8f1b-8cefa8a5b8b4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:54.652123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c546a419-3b6d-43da-8f1b-8cefa8a5b8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:54.682913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:54.758660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c546a419-3b6d-43da-8f1b-8cefa8a5b8b4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:54.835285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:54.835934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:54.836216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:54.836491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:54.838215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:54.913239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:54.955601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:54.957295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5608a7e7-8622-40ae-b3b4-ca2ee581a695 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:54.980811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 672/1331] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:57.255581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46ab74-0f4c-4c80-9e6c-e4ff9a4c0452 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:57.256114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46ab74-0f4c-4c80-9e6c-e4ff9a4c0452 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:57.256312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b46ab74-0f4c-4c80-9e6c-e4ff9a4c0452 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:57.257503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 660/1332] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:57.262601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e119a87-76b4-4834-baa5-203f4222223e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:57.263020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e119a87-76b4-4834-baa5-203f4222223e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:57.263173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e119a87-76b4-4834-baa5-203f4222223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:57.327327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e119a87-76b4-4834-baa5-203f4222223e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:57.328546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e119a87-76b4-4834-baa5-203f4222223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:57.422675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 673/1333] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:53:57.458758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46ab74-0f4c-4c80-9e6c-e4ff9a4c0452 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:53:57.460292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9b46ab74-0f4c-4c80-9e6c-e4ff9a4c0452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:57.529854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:57.563159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b46ab74-0f4c-4c80-9e6c-e4ff9a4c0452 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:53:57.634136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:53:57.634856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:57.635139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:57.635441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:57.637049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:57.741590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:53:57.743013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024294c-3a6e-44ac-b3c0-28908cd957bc service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:53:57.753370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:53:57.762153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 661/1334] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:53:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:53:59.827997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa02de-c1a1-43eb-9916-e87d162ecb68 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:59.828449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa02de-c1a1-43eb-9916-e87d162ecb68 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:59.828557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa02de-c1a1-43eb-9916-e87d162ecb68 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:59.829918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 674/1335] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:53:59.834975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af882dc-b7d7-452c-a0ce-91035f3c4fa0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:53:59.835391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af882dc-b7d7-452c-a0ce-91035f3c4fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:53:59.835573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af882dc-b7d7-452c-a0ce-91035f3c4fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:53:59.900157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6af882dc-b7d7-452c-a0ce-91035f3c4fa0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:53:59.901382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6af882dc-b7d7-452c-a0ce-91035f3c4fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:53:59.994777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 662/1336] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:00.034661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa02de-c1a1-43eb-9916-e87d162ecb68 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:00.036281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f8aa02de-c1a1-43eb-9916-e87d162ecb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:00.076409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:00.139430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa02de-c1a1-43eb-9916-e87d162ecb68 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:00.211587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:00.212239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:00.212526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:00.212801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:00.214705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:00.295303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:00.320766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:00.322200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bc05a-f3b2-488f-b757-039eb9c0c19e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:00.340230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 675/1337] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:02.653958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60e58e-7600-40f7-b481-8c278bd489b4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:02.654475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60e58e-7600-40f7-b481-8c278bd489b4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:02.654475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b60e58e-7600-40f7-b481-8c278bd489b4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:02.655866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 663/1338] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:02.661122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222416-332f-4425-8a92-15fac2228361 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:02.661549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222416-332f-4425-8a92-15fac2228361 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:02.661690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52222416-332f-4425-8a92-15fac2228361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:02.725035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-52222416-332f-4425-8a92-15fac2228361 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:02.726231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-52222416-332f-4425-8a92-15fac2228361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:02.820674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 676/1339] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:02 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:02.862729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60e58e-7600-40f7-b481-8c278bd489b4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:02.864589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3b60e58e-7600-40f7-b481-8c278bd489b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:02.938176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:02.965470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b60e58e-7600-40f7-b481-8c278bd489b4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:03.035158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:03.035822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:03.036109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:03.036379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:03.037987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:03.141299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:03.142707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc0cf30-6b21-4d1b-9a12-499736b6d7a3 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:03.159999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 664/1340] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:07.082516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeea4e9-6eb8-4993-91ab-d5461830d55f None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:07.083061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeea4e9-6eb8-4993-91ab-d5461830d55f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:07.083061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeea4e9-6eb8-4993-91ab-d5461830d55f None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:07.084403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 677/1341] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:07.090403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb91b1f-84d1-4618-ba98-dbbefcb7945a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:07.090845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb91b1f-84d1-4618-ba98-dbbefcb7945a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:07.091128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb91b1f-84d1-4618-ba98-dbbefcb7945a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:07.097598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:07.162183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dfb91b1f-84d1-4618-ba98-dbbefcb7945a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:07.163447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dfb91b1f-84d1-4618-ba98-dbbefcb7945a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:07.267668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 665/1342] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:07.308798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeea4e9-6eb8-4993-91ab-d5461830d55f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:07.311012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cdeea4e9-6eb8-4993-91ab-d5461830d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:07.359666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:07.406173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeea4e9-6eb8-4993-91ab-d5461830d55f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:07.409722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687cbf0-c679-4101-bcc1-0d2784dc0418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:07.410362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687cbf0-c679-4101-bcc1-0d2784dc0418 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:07.410645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687cbf0-c679-4101-bcc1-0d2784dc0418 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:07.410920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687cbf0-c679-4101-bcc1-0d2784dc0418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:07.419632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687cbf0-c679-4101-bcc1-0d2784dc0418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:07.421054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3687cbf0-c679-4101-bcc1-0d2784dc0418 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:07.430226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 678/1343] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:10.106032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb484b-8064-4137-9bb4-4556c4c9b6b2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:10.106541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb484b-8064-4137-9bb4-4556c4c9b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:10.106541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdb484b-8064-4137-9bb4-4556c4c9b6b2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:10.107983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 666/1344] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:10.113320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382144-9bbb-4001-a603-2091d3cf515a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:10.113715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382144-9bbb-4001-a603-2091d3cf515a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:10.114009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45382144-9bbb-4001-a603-2091d3cf515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:10.141749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-45382144-9bbb-4001-a603-2091d3cf515a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:10.142959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-45382144-9bbb-4001-a603-2091d3cf515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:10.226891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 679/1345] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:10.265375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb484b-8064-4137-9bb4-4556c4c9b6b2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:10.277034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdb484b-8064-4137-9bb4-4556c4c9b6b2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:10.280777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1895407-a960-42d5-b07d-87e4376d9176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:10.281527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1895407-a960-42d5-b07d-87e4376d9176 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:10.281883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1895407-a960-42d5-b07d-87e4376d9176 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:10.282232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1895407-a960-42d5-b07d-87e4376d9176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:10.292795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1895407-a960-42d5-b07d-87e4376d9176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:10.294411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1895407-a960-42d5-b07d-87e4376d9176 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:10.305245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 667/1346] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:14.288579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f165f-ad2f-4b34-9e67-c0bb8d38f3ae None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:14.289063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f165f-ad2f-4b34-9e67-c0bb8d38f3ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:14.289158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f165f-ad2f-4b34-9e67-c0bb8d38f3ae None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:14.290615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 680/1347] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:14.295909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2c02-8589-4b48-87f5-b318e88def4f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:14.296358 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2c02-8589-4b48-87f5-b318e88def4f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:14.296518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2c02-8589-4b48-87f5-b318e88def4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:14.316783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-edfe2c02-8589-4b48-87f5-b318e88def4f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:14.318022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-edfe2c02-8589-4b48-87f5-b318e88def4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:14.326796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 668/1348] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:14.364216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f165f-ad2f-4b34-9e67-c0bb8d38f3ae None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:14.375206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f165f-ad2f-4b34-9e67-c0bb8d38f3ae None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:14.378906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3d7918-5dcc-46a6-a665-2e418f388a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:14.379621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d7918-5dcc-46a6-a665-2e418f388a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:14.379911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d7918-5dcc-46a6-a665-2e418f388a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:14.380192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3d7918-5dcc-46a6-a665-2e418f388a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:14.390388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d7918-5dcc-46a6-a665-2e418f388a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:14.391924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3d7918-5dcc-46a6-a665-2e418f388a34 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:14.402052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 681/1349] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:19.462359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09071-23c7-4b52-939c-d95810389d28 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:19.463127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09071-23c7-4b52-939c-d95810389d28 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:19.463221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f09071-23c7-4b52-939c-d95810389d28 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:19.465148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 669/1350] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 05:54:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:19.469658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffd347-6363-4047-802f-1e5a50eaeb74 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:19.470130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffd347-6363-4047-802f-1e5a50eaeb74 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:19.470221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ffd347-6363-4047-802f-1e5a50eaeb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:19.579567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-31ffd347-6363-4047-802f-1e5a50eaeb74 None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:19.598306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31ffd347-6363-4047-802f-1e5a50eaeb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:19.641433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:19.703219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 682/1351] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 05:54:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:23.751680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1767ac0-7f9e-4ea9-85dc-ed9a8b328972 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:23.752244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1767ac0-7f9e-4ea9-85dc-ed9a8b328972 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:23.752244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1767ac0-7f9e-4ea9-85dc-ed9a8b328972 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:23.753929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 670/1352] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 05:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:23.759594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c72440-ea3b-4ec8-b479-650a074b64c1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:23.760039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c72440-ea3b-4ec8-b479-650a074b64c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:23.760459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c72440-ea3b-4ec8-b479-650a074b64c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:23.780818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-13c72440-ea3b-4ec8-b479-650a074b64c1 None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:23.782123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13c72440-ea3b-4ec8-b479-650a074b64c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:23.790898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 683/1353] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 05:54:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:24.311895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=72 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:25.419436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e70ec-f845-4548-929f-f907318a2175 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:25.420081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e70ec-f845-4548-929f-f907318a2175 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:25.420081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01e70ec-f845-4548-929f-f907318a2175 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:25.421619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 671/1354] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 05:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:25.426651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f61c2-109f-4f86-ac20-b0b136764bcf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:25.427076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f61c2-109f-4f86-ac20-b0b136764bcf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:25.427361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4f61c2-109f-4f86-ac20-b0b136764bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:25.446061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ec4f61c2-109f-4f86-ac20-b0b136764bcf None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:25.447235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ec4f61c2-109f-4f86-ac20-b0b136764bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:25.457469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 684/1355] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 05:54:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 05:54:29.026645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154239-d022-4660-a305-93ca3dd881b1 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:29.027649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154239-d022-4660-a305-93ca3dd881b1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:29.028139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be154239-d022-4660-a305-93ca3dd881b1 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:29.034275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 672/1356] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:29.036464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040ebf-cd2b-4333-9c84-889f63f47215 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:29.037298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040ebf-cd2b-4333-9c84-889f63f47215 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:29.037809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80040ebf-cd2b-4333-9c84-889f63f47215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:29.057113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-80040ebf-cd2b-4333-9c84-889f63f47215 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:29.058419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-80040ebf-cd2b-4333-9c84-889f63f47215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:29.066634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 685/1357] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:29.124134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148bd7b-7173-4abc-839c-228f428f1b1f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:29.124736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148bd7b-7173-4abc-839c-228f428f1b1f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:29.125246 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148bd7b-7173-4abc-839c-228f428f1b1f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:29.127637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 673/1358] 10.222.0.45 () {54 vars in 961 bytes} [Mon Jun 20 05:54:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:29.138378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39698488-aa1d-43ec-9e38-4f2943a3823d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:29.140042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39698488-aa1d-43ec-9e38-4f2943a3823d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:29.140575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39698488-aa1d-43ec-9e38-4f2943a3823d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:29.193840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-39698488-aa1d-43ec-9e38-4f2943a3823d None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:29.195990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-39698488-aa1d-43ec-9e38-4f2943a3823d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:29.284451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 686/1359] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Jun 20 05:54:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:29.291796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148bd7b-7173-4abc-839c-228f428f1b1f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:29.306629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148bd7b-7173-4abc-839c-228f428f1b1f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:29.312059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37b460-1058-49f6-ae87-e9ab420c89c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:29.312702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37b460-1058-49f6-ae87-e9ab420c89c0 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:29.312984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37b460-1058-49f6-ae87-e9ab420c89c0 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:29.313259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37b460-1058-49f6-ae87-e9ab420c89c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:29.322400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37b460-1058-49f6-ae87-e9ab420c89c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:29.324032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37b460-1058-49f6-ae87-e9ab420c89c0 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:29.347695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 674/1360] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:31.564986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2cb3d-781b-4878-86e8-52efe70a3179 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:31.565553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2cb3d-781b-4878-86e8-52efe70a3179 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:31.565553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2cb3d-781b-4878-86e8-52efe70a3179 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:31.567068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 687/1361] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:31.572704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a16f-2039-40ec-bb5d-9ccf2aceac79 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:31.573096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a16f-2039-40ec-bb5d-9ccf2aceac79 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:31.573278 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5a16f-2039-40ec-bb5d-9ccf2aceac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:31.591874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f5b5a16f-2039-40ec-bb5d-9ccf2aceac79 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:31.593205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f5b5a16f-2039-40ec-bb5d-9ccf2aceac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:31.601345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 675/1362] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:31.645443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2cb3d-781b-4878-86e8-52efe70a3179 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:31.659641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2cb3d-781b-4878-86e8-52efe70a3179 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:31.665867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98a221f-bf82-4548-a759-3ae7b6e93c62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:31.666513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a221f-bf82-4548-a759-3ae7b6e93c62 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:31.666799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a221f-bf82-4548-a759-3ae7b6e93c62 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:31.667091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98a221f-bf82-4548-a759-3ae7b6e93c62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:31.675753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a221f-bf82-4548-a759-3ae7b6e93c62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:31.677346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98a221f-bf82-4548-a759-3ae7b6e93c62 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:31.687286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 688/1363] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:34.053115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88557f5f-41b2-4107-bf27-6b520c8c2623 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:34.053737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88557f5f-41b2-4107-bf27-6b520c8c2623 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:34.053737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88557f5f-41b2-4107-bf27-6b520c8c2623 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:34.055150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 676/1364] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:34.060675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f36c73-2b43-4351-bb0c-0f20cd5e9a55 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:34.061167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f36c73-2b43-4351-bb0c-0f20cd5e9a55 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:34.061262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f36c73-2b43-4351-bb0c-0f20cd5e9a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:34.081122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-33f36c73-2b43-4351-bb0c-0f20cd5e9a55 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:34.082223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-33f36c73-2b43-4351-bb0c-0f20cd5e9a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:34.092156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 689/1365] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:34.137452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f167f9-4136-4190-916a-5d6c3108b5c3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:34.137764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f167f9-4136-4190-916a-5d6c3108b5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:34.138048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f167f9-4136-4190-916a-5d6c3108b5c3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:34.139447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 677/1366] 10.222.0.45 () {54 vars in 961 bytes} [Mon Jun 20 05:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:34.144809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac311-8679-47c5-92d9-ff8d1108c873 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:34.145164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac311-8679-47c5-92d9-ff8d1108c873 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:34.145507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdac311-8679-47c5-92d9-ff8d1108c873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:34.164082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8bdac311-8679-47c5-92d9-ff8d1108c873 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:34.165325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8bdac311-8679-47c5-92d9-ff8d1108c873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:34.174348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 690/1367] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Jun 20 05:54:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:34.182125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f167f9-4136-4190-916a-5d6c3108b5c3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:34.196478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f167f9-4136-4190-916a-5d6c3108b5c3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:34.202003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc79120-e283-468a-9fc0-b2d8bbf677a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:34.202654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc79120-e283-468a-9fc0-b2d8bbf677a2 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:34.203116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc79120-e283-468a-9fc0-b2d8bbf677a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:34.203425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc79120-e283-468a-9fc0-b2d8bbf677a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:34.211849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc79120-e283-468a-9fc0-b2d8bbf677a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:34.213652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc79120-e283-468a-9fc0-b2d8bbf677a2 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:34.223061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 678/1368] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:36.353896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0b4cc-ef7c-4d59-ac13-4a11856fca8a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:36.354442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0b4cc-ef7c-4d59-ac13-4a11856fca8a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:36.354442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff0b4cc-ef7c-4d59-ac13-4a11856fca8a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:36.355879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 691/1369] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:36.361225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7446b7-ff2d-426f-9049-d97d2ccd322e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:36.361648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7446b7-ff2d-426f-9049-d97d2ccd322e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:36.361786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7446b7-ff2d-426f-9049-d97d2ccd322e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:36.381554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ae7446b7-ff2d-426f-9049-d97d2ccd322e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:36.383382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ae7446b7-ff2d-426f-9049-d97d2ccd322e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:36.395282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 679/1370] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:36.436797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0b4cc-ef7c-4d59-ac13-4a11856fca8a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:36.452320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff0b4cc-ef7c-4d59-ac13-4a11856fca8a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:36.458291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943eb3a-dac1-42ab-9894-c0b3d47e2419 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:36.459161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943eb3a-dac1-42ab-9894-c0b3d47e2419 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:36.459640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943eb3a-dac1-42ab-9894-c0b3d47e2419 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:36.460089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943eb3a-dac1-42ab-9894-c0b3d47e2419 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:36.469798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943eb3a-dac1-42ab-9894-c0b3d47e2419 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:36.471622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f943eb3a-dac1-42ab-9894-c0b3d47e2419 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:36.481581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 692/1371] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:37.022333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98f5f5-6353-4cbf-ae64-b1dc40d10ae4 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:37.022655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98f5f5-6353-4cbf-ae64-b1dc40d10ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:37.023111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98f5f5-6353-4cbf-ae64-b1dc40d10ae4 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:37.024689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 680/1372] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:37.030358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d156ea0-4d91-4b65-aa98-e283ececb8a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:37.030653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d156ea0-4d91-4b65-aa98-e283ececb8a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:37.030934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d156ea0-4d91-4b65-aa98-e283ececb8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:37.056537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4d156ea0-4d91-4b65-aa98-e283ececb8a9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:37.057772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4d156ea0-4d91-4b65-aa98-e283ececb8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:37.065932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 693/1373] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:37.373894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e159c-c88d-453a-8490-c9903387183c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:37.374373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e159c-c88d-453a-8490-c9903387183c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:37.374511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e159c-c88d-453a-8490-c9903387183c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:37.376089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 681/1374] 10.222.0.45 () {54 vars in 963 bytes} [Mon Jun 20 05:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:37.381267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d60e8-a282-458f-bbe0-891e6c52537d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:37.381692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d60e8-a282-458f-bbe0-891e6c52537d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:37.381921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d60e8-a282-458f-bbe0-891e6c52537d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:37.388979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:37.434560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-829d60e8-a282-458f-bbe0-891e6c52537d None None] MFA Rules not processed for user `00892de83c5444c7a38b921615b2f5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:37.435789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-829d60e8-a282-458f-bbe0-891e6c52537d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:37.502087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 694/1375] 10.222.0.45 () {58 vars in 1060 bytes} [Mon Jun 20 05:54:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:37.509791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e159c-c88d-453a-8490-c9903387183c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:37.520544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e159c-c88d-453a-8490-c9903387183c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:37.524658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98940f4a-4e17-4068-b951-ae7095d52f74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:37.525310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98940f4a-4e17-4068-b951-ae7095d52f74 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:37.525588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98940f4a-4e17-4068-b951-ae7095d52f74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:37.526061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98940f4a-4e17-4068-b951-ae7095d52f74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:37.535560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98940f4a-4e17-4068-b951-ae7095d52f74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:37.537153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98940f4a-4e17-4068-b951-ae7095d52f74 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:37.547736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 682/1376] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:38.584495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf743f2-1dd0-427f-9c08-1f0a521c8d8e None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:38.585062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf743f2-1dd0-427f-9c08-1f0a521c8d8e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:38.585062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf743f2-1dd0-427f-9c08-1f0a521c8d8e None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:38.586367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 695/1377] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:38.591932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe401b6-707a-41c0-9f44-872586c7923c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:38.592372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe401b6-707a-41c0-9f44-872586c7923c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:38.592680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe401b6-707a-41c0-9f44-872586c7923c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:38.610707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8fe401b6-707a-41c0-9f44-872586c7923c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:38.611968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8fe401b6-707a-41c0-9f44-872586c7923c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:38.620700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 683/1378] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:38.665303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf743f2-1dd0-427f-9c08-1f0a521c8d8e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:38.680408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf743f2-1dd0-427f-9c08-1f0a521c8d8e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:38.685874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e20e8d-f577-41ef-bdbe-d814421bdd80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:38.686508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20e8d-f577-41ef-bdbe-d814421bdd80 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:38.686795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20e8d-f577-41ef-bdbe-d814421bdd80 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:38.687074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e20e8d-f577-41ef-bdbe-d814421bdd80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:38.696763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20e8d-f577-41ef-bdbe-d814421bdd80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:38.698718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e20e8d-f577-41ef-bdbe-d814421bdd80 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:38.708675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 696/1379] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:40.754587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbbe83-c29a-4394-a770-106af3bc4376 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:40.755108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbbe83-c29a-4394-a770-106af3bc4376 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:40.755220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbbe83-c29a-4394-a770-106af3bc4376 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:40.756540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 684/1380] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:40.762364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e6c74-1018-461e-b8e8-279d1efacb9f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:40.762789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e6c74-1018-461e-b8e8-279d1efacb9f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:40.762884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e6c74-1018-461e-b8e8-279d1efacb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:40.787126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b93e6c74-1018-461e-b8e8-279d1efacb9f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:40.788362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b93e6c74-1018-461e-b8e8-279d1efacb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:40.796417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 697/1381] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:40.842416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbbe83-c29a-4394-a770-106af3bc4376 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:40.857164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbbe83-c29a-4394-a770-106af3bc4376 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:40.867058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a6eee1-1368-4f29-8625-368b420e06b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:40.867942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6eee1-1368-4f29-8625-368b420e06b6 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:40.868286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6eee1-1368-4f29-8625-368b420e06b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:40.868572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a6eee1-1368-4f29-8625-368b420e06b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:40.878042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6eee1-1368-4f29-8625-368b420e06b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:40.882561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a6eee1-1368-4f29-8625-368b420e06b6 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:40.902146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 685/1382] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:43.000342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ae2c9-af7e-4007-bb3c-a8f72f0502ed None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:43.001294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ae2c9-af7e-4007-bb3c-a8f72f0502ed None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:43.001592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ae2c9-af7e-4007-bb3c-a8f72f0502ed None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:43.003167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 698/1383] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:43.008850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe5a79-4b87-4d20-88fa-1277bbb38d1f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:43.009302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe5a79-4b87-4d20-88fa-1277bbb38d1f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:43.009402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe5a79-4b87-4d20-88fa-1277bbb38d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:43.028215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-86fe5a79-4b87-4d20-88fa-1277bbb38d1f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:43.029464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-86fe5a79-4b87-4d20-88fa-1277bbb38d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:43.039010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 686/1384] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:43.082601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ae2c9-af7e-4007-bb3c-a8f72f0502ed None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:43.098793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ae2c9-af7e-4007-bb3c-a8f72f0502ed None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:43.104424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f426bdd0-c840-47ee-b090-17eb6cf98da7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:43.105199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426bdd0-c840-47ee-b090-17eb6cf98da7 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:43.105591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426bdd0-c840-47ee-b090-17eb6cf98da7 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:43.105995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f426bdd0-c840-47ee-b090-17eb6cf98da7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:43.115624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426bdd0-c840-47ee-b090-17eb6cf98da7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:43.117355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f426bdd0-c840-47ee-b090-17eb6cf98da7 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:43.129945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 699/1385] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:45.175458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3221e-6d79-44a2-bbe6-cf74d3097496 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:45.176021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3221e-6d79-44a2-bbe6-cf74d3097496 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:45.176128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3221e-6d79-44a2-bbe6-cf74d3097496 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:45.177481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 687/1386] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:45.182925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933728e0-e7bf-4b7c-b32d-4a2e375f0684 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:45.183372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933728e0-e7bf-4b7c-b32d-4a2e375f0684 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:45.183559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933728e0-e7bf-4b7c-b32d-4a2e375f0684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:45.202438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-933728e0-e7bf-4b7c-b32d-4a2e375f0684 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:45.203705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-933728e0-e7bf-4b7c-b32d-4a2e375f0684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:45.212789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 700/1387] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 05:54:45.257876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3221e-6d79-44a2-bbe6-cf74d3097496 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:45.273359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3221e-6d79-44a2-bbe6-cf74d3097496 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:45.278865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9fc5a-c0ea-4cab-9b8b-49e9f487e48c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:45.279571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9fc5a-c0ea-4cab-9b8b-49e9f487e48c service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:45.279861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9fc5a-c0ea-4cab-9b8b-49e9f487e48c service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:45.280144 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9fc5a-c0ea-4cab-9b8b-49e9f487e48c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:45.288555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9fc5a-c0ea-4cab-9b8b-49e9f487e48c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:45.290137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9fc5a-c0ea-4cab-9b8b-49e9f487e48c service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:45.299727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 688/1388] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:47.363628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fcc3f-af62-417b-9e9c-0793ec24ec53 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:47.364167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fcc3f-af62-417b-9e9c-0793ec24ec53 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:47.364256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9fcc3f-af62-417b-9e9c-0793ec24ec53 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:47.365574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 701/1389] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:47.371457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab2b4-a08d-454a-b1ed-1ded4fe82a82 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:47.371857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab2b4-a08d-454a-b1ed-1ded4fe82a82 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:47.372061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcab2b4-a08d-454a-b1ed-1ded4fe82a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:47.390676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8dcab2b4-a08d-454a-b1ed-1ded4fe82a82 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:47.392349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8dcab2b4-a08d-454a-b1ed-1ded4fe82a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:47.402515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 689/1390] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:47.455367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fcc3f-af62-417b-9e9c-0793ec24ec53 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:47.473324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9fcc3f-af62-417b-9e9c-0793ec24ec53 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:47.480607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb06e81-cc93-44b6-a861-279bd249bbb9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:47.481644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06e81-cc93-44b6-a861-279bd249bbb9 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:47.482197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06e81-cc93-44b6-a861-279bd249bbb9 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:47.482573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb06e81-cc93-44b6-a861-279bd249bbb9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:47.493364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb06e81-cc93-44b6-a861-279bd249bbb9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:47.495410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb06e81-cc93-44b6-a861-279bd249bbb9 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:47.506909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 702/1391] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:49.572057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cdcba-2e07-4e1d-9306-138e65eabd48 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:49.572598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cdcba-2e07-4e1d-9306-138e65eabd48 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:49.572720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81cdcba-2e07-4e1d-9306-138e65eabd48 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:49.574002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 690/1392] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:49.579449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79047-34da-438e-b0e5-a251856d2937 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:49.579857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79047-34da-438e-b0e5-a251856d2937 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:49.580044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d79047-34da-438e-b0e5-a251856d2937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:49.598658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-85d79047-34da-438e-b0e5-a251856d2937 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:49.599899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-85d79047-34da-438e-b0e5-a251856d2937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:49.607738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 703/1393] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:49.653822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cdcba-2e07-4e1d-9306-138e65eabd48 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:49.669062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81cdcba-2e07-4e1d-9306-138e65eabd48 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:49.674541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b25ee-d14a-4100-b318-683bd573afe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:49.675180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b25ee-d14a-4100-b318-683bd573afe4 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:49.675492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b25ee-d14a-4100-b318-683bd573afe4 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:49.675767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b25ee-d14a-4100-b318-683bd573afe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:49.685142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b25ee-d14a-4100-b318-683bd573afe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:49.686707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b25ee-d14a-4100-b318-683bd573afe4 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:49.696321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 691/1394] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:51.753256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1dba7-d275-4aff-979c-dfec4bfee5e1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:51.753801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1dba7-d275-4aff-979c-dfec4bfee5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:51.753801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1dba7-d275-4aff-979c-dfec4bfee5e1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:51.755179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 704/1395] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:51.761024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd61b9-8c97-41e2-959d-adba4b4df065 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:51.761455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd61b9-8c97-41e2-959d-adba4b4df065 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:51.761592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd61b9-8c97-41e2-959d-adba4b4df065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:51.779374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9edd61b9-8c97-41e2-959d-adba4b4df065 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:51.780580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9edd61b9-8c97-41e2-959d-adba4b4df065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:51.788635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 692/1396] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:51.833478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1dba7-d275-4aff-979c-dfec4bfee5e1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:51.848243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1dba7-d275-4aff-979c-dfec4bfee5e1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:51.853534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eac3109-59da-45bc-89bf-60a470fdb836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:51.854210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac3109-59da-45bc-89bf-60a470fdb836 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:51.854481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac3109-59da-45bc-89bf-60a470fdb836 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:51.854763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eac3109-59da-45bc-89bf-60a470fdb836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:51.862816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac3109-59da-45bc-89bf-60a470fdb836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:51.864428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eac3109-59da-45bc-89bf-60a470fdb836 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:51.885450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 705/1397] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:53.085212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3ef2-0039-4be1-baa8-79c0430cbbf3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.085699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3ef2-0039-4be1-baa8-79c0430cbbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.085699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447c3ef2-0039-4be1-baa8-79c0430cbbf3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.087362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 693/1398] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:53.092591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c4c4-54da-48a2-be42-ff2535b13549 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.093039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c4c4-54da-48a2-be42-ff2535b13549 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.093180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93c4c4-54da-48a2-be42-ff2535b13549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.112039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3c93c4c4-54da-48a2-be42-ff2535b13549 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:53.113283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3c93c4c4-54da-48a2-be42-ff2535b13549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:53.121726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 706/1399] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:53.176798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadee67-fb30-454b-838e-a2888f4aece6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.177094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadee67-fb30-454b-838e-a2888f4aece6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.177401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadee67-fb30-454b-838e-a2888f4aece6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.179356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 694/1400] 10.222.0.45 () {54 vars in 962 bytes} [Mon Jun 20 05:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:53.184549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c610be-0737-458a-95ab-75198a89590b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.184914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c610be-0737-458a-95ab-75198a89590b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.185265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c610be-0737-458a-95ab-75198a89590b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.232866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f0c610be-0737-458a-95ab-75198a89590b None None] MFA Rules not processed for user `89c777401892485099d5448c506debb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:53.234081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f0c610be-0737-458a-95ab-75198a89590b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:53.266854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:53.294098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 707/1401] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Jun 20 05:54:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:53.301753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadee67-fb30-454b-838e-a2888f4aece6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:53.312861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadee67-fb30-454b-838e-a2888f4aece6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:53.316384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd35d5-c926-411f-ab1e-c1bcf1a8a7a4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:53.317023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd35d5-c926-411f-ab1e-c1bcf1a8a7a4 service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.317328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd35d5-c926-411f-ab1e-c1bcf1a8a7a4 service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.317610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd35d5-c926-411f-ab1e-c1bcf1a8a7a4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.327149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd35d5-c926-411f-ab1e-c1bcf1a8a7a4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:53.328601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd35d5-c926-411f-ab1e-c1bcf1a8a7a4 service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:53.338369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 695/1402] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:53.953246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386df89-e6a7-4d64-96db-c66974202293 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.953669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386df89-e6a7-4d64-96db-c66974202293 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.954025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8386df89-e6a7-4d64-96db-c66974202293 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.955662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 708/1403] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:53.961368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e94f6-331d-467f-ba72-3ead4e8e2be1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:53.961735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e94f6-331d-467f-ba72-3ead4e8e2be1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:53.962283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70e94f6-331d-467f-ba72-3ead4e8e2be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:53.980106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f70e94f6-331d-467f-ba72-3ead4e8e2be1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:53.981315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f70e94f6-331d-467f-ba72-3ead4e8e2be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:53.989235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 696/1404] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:54:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:54.034474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386df89-e6a7-4d64-96db-c66974202293 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:54.049511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8386df89-e6a7-4d64-96db-c66974202293 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:54.055106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ce0d5-ca52-4596-8e2e-4288be808acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:54.055781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce0d5-ca52-4596-8e2e-4288be808acd service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:54.056069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce0d5-ca52-4596-8e2e-4288be808acd service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:54.056364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ce0d5-ca52-4596-8e2e-4288be808acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:54.056506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f39c-8511-4022-a48d-bdbd511d54fa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:54.056938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f39c-8511-4022-a48d-bdbd511d54fa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:54.057307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181f39c-8511-4022-a48d-bdbd511d54fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:54.064855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ce0d5-ca52-4596-8e2e-4288be808acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:54.066463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76ce0d5-ca52-4596-8e2e-4288be808acd service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:54.075831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 709/1405] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:54:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:54:54.107746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3181f39c-8511-4022-a48d-bdbd511d54fa None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:54.111919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3181f39c-8511-4022-a48d-bdbd511d54fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:54.171338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:54:54.183180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 697/1406] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 05:54:54.228614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b1a4-bdba-48b4-8d60-76ef65fa421a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:54.229196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b1a4-bdba-48b4-8d60-76ef65fa421a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:54.229196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5b1a4-bdba-48b4-8d60-76ef65fa421a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:54.230598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 710/1407] 10.222.0.45 () {54 vars in 942 bytes} [Mon Jun 20 05:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:54:54.237663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089721-54e2-4e75-880f-33c5b74049fa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:54.238032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089721-54e2-4e75-880f-33c5b74049fa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:54.238407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24089721-54e2-4e75-880f-33c5b74049fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:54.290056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-24089721-54e2-4e75-880f-33c5b74049fa None None] MFA Rules not processed for user `a7ac2b5986a74ea9923fa0593e58a6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:54:54.291866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-24089721-54e2-4e75-880f-33c5b74049fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:54:54.367347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 698/1408] 10.222.0.45 () {58 vars in 1039 bytes} [Mon Jun 20 05:54:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:54:54.376534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b1a4-bdba-48b4-8d60-76ef65fa421a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:54:54.390007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5b1a4-bdba-48b4-8d60-76ef65fa421a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:54:54.394879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28283f6e-da22-442f-b2bd-9607dc0f6a4c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:54:54.395543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28283f6e-da22-442f-b2bd-9607dc0f6a4c service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:54:54.395827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28283f6e-da22-442f-b2bd-9607dc0f6a4c service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:54:54.396128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28283f6e-da22-442f-b2bd-9607dc0f6a4c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:54:54.404540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28283f6e-da22-442f-b2bd-9607dc0f6a4c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:54:54.406125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28283f6e-da22-442f-b2bd-9607dc0f6a4c service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:54:54.414097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 711/1409] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 05:54:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 05:55:04.364035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=47 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:55:04.410967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=22 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:55:16.598127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ff16c-0e75-4b29-8d6d-031117d31795 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:16.599307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ff16c-0e75-4b29-8d6d-031117d31795 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:16.599431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ff16c-0e75-4b29-8d6d-031117d31795 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:16.600735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 699/1410] 10.222.0.45 () {54 vars in 918 bytes} [Mon Jun 20 05:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:16.605777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f60f1-d6a6-4f50-85ae-00f0f771f2ad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:16.606183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f60f1-d6a6-4f50-85ae-00f0f771f2ad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:16.606283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f60f1-d6a6-4f50-85ae-00f0f771f2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:16.623991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-415f60f1-d6a6-4f50-85ae-00f0f771f2ad None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:16.625177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-415f60f1-d6a6-4f50-85ae-00f0f771f2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:16.634161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 712/1411] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Jun 20 05:55:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 05:55:16.648782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cbaf2-2d94-4e42-a7c8-a90a8575b0a8 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:16.649073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cbaf2-2d94-4e42-a7c8-a90a8575b0a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:16.649353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4cbaf2-2d94-4e42-a7c8-a90a8575b0a8 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:16.650687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 700/1412] 10.222.0.45 () {54 vars in 918 bytes} [Mon Jun 20 05:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:16.654992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dfa02-a944-40eb-bb97-4d7c41960a35 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:16.655312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dfa02-a944-40eb-bb97-4d7c41960a35 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:16.655595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975dfa02-a944-40eb-bb97-4d7c41960a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:16.672659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-975dfa02-a944-40eb-bb97-4d7c41960a35 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:16.673815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-975dfa02-a944-40eb-bb97-4d7c41960a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:16.682679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 713/1413] 10.222.0.45 () {58 vars in 1015 bytes} [Mon Jun 20 05:55:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:19.176689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc3db0-3d29-434f-8168-cfc8bb037ab3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:19.177219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc3db0-3d29-434f-8168-cfc8bb037ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:19.177333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dc3db0-3d29-434f-8168-cfc8bb037ab3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:19.178628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 701/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:19.183755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb6182-8218-4dea-b59b-5b0c92b03c8d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:19.184205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb6182-8218-4dea-b59b-5b0c92b03c8d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:19.184316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb6182-8218-4dea-b59b-5b0c92b03c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:19.202222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-06cb6182-8218-4dea-b59b-5b0c92b03c8d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:19.203449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-06cb6182-8218-4dea-b59b-5b0c92b03c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:19.212077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 714/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:19.254985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc3db0-3d29-434f-8168-cfc8bb037ab3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:19.269428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dc3db0-3d29-434f-8168-cfc8bb037ab3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:19.275453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aecd81-46e6-4dd0-9b6a-bb72d62057e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:19.281881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aecd81-46e6-4dd0-9b6a-bb72d62057e8 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:19.282208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aecd81-46e6-4dd0-9b6a-bb72d62057e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:19.282966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aecd81-46e6-4dd0-9b6a-bb72d62057e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:19.291190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aecd81-46e6-4dd0-9b6a-bb72d62057e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:19.292784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aecd81-46e6-4dd0-9b6a-bb72d62057e8 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:19.302517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 702/1416] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:20.203418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e73f2-6218-448c-8d1d-fdf26caefc0d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:20.203949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e73f2-6218-448c-8d1d-fdf26caefc0d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:20.203949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5e73f2-6218-448c-8d1d-fdf26caefc0d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:20.205350 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 715/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:20.211053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d798937-cf37-442d-b77b-56af22b04cd7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:20.211759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d798937-cf37-442d-b77b-56af22b04cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:20.211922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d798937-cf37-442d-b77b-56af22b04cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:20.230024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3d798937-cf37-442d-b77b-56af22b04cd7 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:20.231232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d798937-cf37-442d-b77b-56af22b04cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:20.239042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 703/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:20.390872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e73f2-6218-448c-8d1d-fdf26caefc0d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:20.405485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5e73f2-6218-448c-8d1d-fdf26caefc0d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:20.410871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511678-62a6-4df6-9cf0-c305135822ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:20.411542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511678-62a6-4df6-9cf0-c305135822ae service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:20.411830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511678-62a6-4df6-9cf0-c305135822ae service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:20.412111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511678-62a6-4df6-9cf0-c305135822ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:20.420047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14511678-62a6-4df6-9cf0-c305135822ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:20.421628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14511678-62a6-4df6-9cf0-c305135822ae service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:20.430837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 716/1419] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:21.305333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db022a1c-9fff-4ac8-a286-4c06e5d07a1b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:21.305907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db022a1c-9fff-4ac8-a286-4c06e5d07a1b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:21.305907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db022a1c-9fff-4ac8-a286-4c06e5d07a1b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:21.307307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 704/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:21.312060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c1c65-f1ed-42db-8a14-9a30bfaa4c73 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:21.312485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c1c65-f1ed-42db-8a14-9a30bfaa4c73 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:21.312629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c1c65-f1ed-42db-8a14-9a30bfaa4c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:21.330708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-da2c1c65-f1ed-42db-8a14-9a30bfaa4c73 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:21.331931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-da2c1c65-f1ed-42db-8a14-9a30bfaa4c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:21.340507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 717/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:21.489639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db022a1c-9fff-4ac8-a286-4c06e5d07a1b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:21.503718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db022a1c-9fff-4ac8-a286-4c06e5d07a1b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:21.509572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee48ed4-f8b0-4d3b-8160-a6f83e88feda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:21.510208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48ed4-f8b0-4d3b-8160-a6f83e88feda service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:21.510495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48ed4-f8b0-4d3b-8160-a6f83e88feda service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:21.510776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee48ed4-f8b0-4d3b-8160-a6f83e88feda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:21.518728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee48ed4-f8b0-4d3b-8160-a6f83e88feda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:21.520919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee48ed4-f8b0-4d3b-8160-a6f83e88feda service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:21.530149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 705/1422] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:22.377707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6266a03-ae35-482c-b511-24748e4ded7a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:22.378138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6266a03-ae35-482c-b511-24748e4ded7a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:22.378263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6266a03-ae35-482c-b511-24748e4ded7a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:22.379714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 718/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:22.384678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea31ad0-ca0f-44ca-b775-9e9aba36f793 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:22.385086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea31ad0-ca0f-44ca-b775-9e9aba36f793 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:22.385248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea31ad0-ca0f-44ca-b775-9e9aba36f793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:22.403722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7ea31ad0-ca0f-44ca-b775-9e9aba36f793 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:22.404912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ea31ad0-ca0f-44ca-b775-9e9aba36f793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:22.413279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 706/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:22.560677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6266a03-ae35-482c-b511-24748e4ded7a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:22.574908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6266a03-ae35-482c-b511-24748e4ded7a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:22.580399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b773d6fa-1388-47d4-b4a0-53dd7dd384b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:22.581029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d6fa-1388-47d4-b4a0-53dd7dd384b6 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:22.581336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d6fa-1388-47d4-b4a0-53dd7dd384b6 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:22.581617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b773d6fa-1388-47d4-b4a0-53dd7dd384b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:22.590340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d6fa-1388-47d4-b4a0-53dd7dd384b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:22.591951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b773d6fa-1388-47d4-b4a0-53dd7dd384b6 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:22.601681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 719/1425] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:23.467910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6d47f-5f12-4d23-aaa4-94f6ba86b19c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:23.468523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6d47f-5f12-4d23-aaa4-94f6ba86b19c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:23.468523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a6d47f-5f12-4d23-aaa4-94f6ba86b19c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:23.469911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 707/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:23.475241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027db63-f84f-440e-a8c5-5e5b7881d4bf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:23.475690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027db63-f84f-440e-a8c5-5e5b7881d4bf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:23.475828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027db63-f84f-440e-a8c5-5e5b7881d4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:23.494253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c027db63-f84f-440e-a8c5-5e5b7881d4bf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:23.495477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c027db63-f84f-440e-a8c5-5e5b7881d4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:23.503098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 720/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:23.650951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6d47f-5f12-4d23-aaa4-94f6ba86b19c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:23.665691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a6d47f-5f12-4d23-aaa4-94f6ba86b19c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:23.671017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5ac6e1-a665-4666-86bf-deaacf612bc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:23.671678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac6e1-a665-4666-86bf-deaacf612bc7 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:23.671964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac6e1-a665-4666-86bf-deaacf612bc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:23.672239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5ac6e1-a665-4666-86bf-deaacf612bc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:23.680965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac6e1-a665-4666-86bf-deaacf612bc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:23.682553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5ac6e1-a665-4666-86bf-deaacf612bc7 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:23.692002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 708/1428] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:24.498707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d3491-fae8-4657-9c3d-6ab56e7f4641 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:24.499290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d3491-fae8-4657-9c3d-6ab56e7f4641 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:24.499290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d3491-fae8-4657-9c3d-6ab56e7f4641 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:24.500670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 721/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:24.506135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f9fe-0461-44b7-a369-e7c4e0432305 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:24.506567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f9fe-0461-44b7-a369-e7c4e0432305 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:24.506701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc8f9fe-0461-44b7-a369-e7c4e0432305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:24.524831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5fc8f9fe-0461-44b7-a369-e7c4e0432305 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:24.526035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5fc8f9fe-0461-44b7-a369-e7c4e0432305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:24.533909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 709/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:24.684952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d3491-fae8-4657-9c3d-6ab56e7f4641 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:24.699815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d3491-fae8-4657-9c3d-6ab56e7f4641 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:24.705183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ae46f7-42e3-49a7-a7ab-c6ee06fed698 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:24.705820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae46f7-42e3-49a7-a7ab-c6ee06fed698 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:24.706109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae46f7-42e3-49a7-a7ab-c6ee06fed698 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:24.706390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ae46f7-42e3-49a7-a7ab-c6ee06fed698 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:24.714678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae46f7-42e3-49a7-a7ab-c6ee06fed698 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:24.716296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ae46f7-42e3-49a7-a7ab-c6ee06fed698 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:24.725782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 722/1431] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:25.517430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440067e2-66c7-4818-9e6b-6995091d392b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:25.517997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440067e2-66c7-4818-9e6b-6995091d392b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:25.517997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440067e2-66c7-4818-9e6b-6995091d392b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:25.519372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 710/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:25.524802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7a36-9bd3-475d-ba9e-161724dd03fa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:25.525391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7a36-9bd3-475d-ba9e-161724dd03fa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:25.525391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d7a36-9bd3-475d-ba9e-161724dd03fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:25.543568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-709d7a36-9bd3-475d-ba9e-161724dd03fa None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:25.544771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-709d7a36-9bd3-475d-ba9e-161724dd03fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:25.552514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 723/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:25.702275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440067e2-66c7-4818-9e6b-6995091d392b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:25.716594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440067e2-66c7-4818-9e6b-6995091d392b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:25.721906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41271de4-2b1c-4d5d-9b0b-2217184aa034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:25.722530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41271de4-2b1c-4d5d-9b0b-2217184aa034 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:25.722812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41271de4-2b1c-4d5d-9b0b-2217184aa034 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:25.723646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41271de4-2b1c-4d5d-9b0b-2217184aa034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:25.731640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41271de4-2b1c-4d5d-9b0b-2217184aa034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:25.733523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41271de4-2b1c-4d5d-9b0b-2217184aa034 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:25.744009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 711/1434] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:26.537343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fc532-87a2-49dc-80db-69635bde048a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:26.537891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fc532-87a2-49dc-80db-69635bde048a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:26.537891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525fc532-87a2-49dc-80db-69635bde048a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:26.539271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 724/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:26.544058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f39d38-e523-4656-a111-adda0db1e2dd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:26.544506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f39d38-e523-4656-a111-adda0db1e2dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:26.544614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f39d38-e523-4656-a111-adda0db1e2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:26.562527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-00f39d38-e523-4656-a111-adda0db1e2dd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:26.563738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-00f39d38-e523-4656-a111-adda0db1e2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:26.571311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 712/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:26.721970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fc532-87a2-49dc-80db-69635bde048a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:26.736387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525fc532-87a2-49dc-80db-69635bde048a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:26.741780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854d63cb-6d1e-4e06-ad28-ae01b83a2233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:26.742412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d63cb-6d1e-4e06-ad28-ae01b83a2233 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:26.742695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d63cb-6d1e-4e06-ad28-ae01b83a2233 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:26.742971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854d63cb-6d1e-4e06-ad28-ae01b83a2233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:26.750959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d63cb-6d1e-4e06-ad28-ae01b83a2233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:26.752568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854d63cb-6d1e-4e06-ad28-ae01b83a2233 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:26.762296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 725/1437] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:27.557794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da976038-b481-45c8-b967-73bd9f9e97a2 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:27.558339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da976038-b481-45c8-b967-73bd9f9e97a2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:27.558339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da976038-b481-45c8-b967-73bd9f9e97a2 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:27.559752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 713/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:27.564559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a83a-8907-40ad-a9a2-2a737c5e49cb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:27.564952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a83a-8907-40ad-a9a2-2a737c5e49cb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:27.565120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7a83a-8907-40ad-a9a2-2a737c5e49cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:27.583291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4ce7a83a-8907-40ad-a9a2-2a737c5e49cb None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:27.584499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4ce7a83a-8907-40ad-a9a2-2a737c5e49cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:27.592470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 726/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:55:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:27.742626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da976038-b481-45c8-b967-73bd9f9e97a2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:27.757253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da976038-b481-45c8-b967-73bd9f9e97a2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:27.762760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccdfbab-b766-4057-98ea-43cd827a5b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:27.763444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdfbab-b766-4057-98ea-43cd827a5b0c service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:27.763733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdfbab-b766-4057-98ea-43cd827a5b0c service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:27.764043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccdfbab-b766-4057-98ea-43cd827a5b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:27.772257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdfbab-b766-4057-98ea-43cd827a5b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:27.773823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccdfbab-b766-4057-98ea-43cd827a5b0c service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:27.783059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 714/1440] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:29.700162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5891-078b-4e16-95c6-b5a0b4c2d6df None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:29.700699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5891-078b-4e16-95c6-b5a0b4c2d6df None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:29.700820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b5891-078b-4e16-95c6-b5a0b4c2d6df None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:29.702236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 727/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:29.707436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17902-19fb-4460-908f-7eeb6643ea5f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:29.708442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17902-19fb-4460-908f-7eeb6643ea5f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:29.708588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba17902-19fb-4460-908f-7eeb6643ea5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:29.726043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dba17902-19fb-4460-908f-7eeb6643ea5f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:29.727236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dba17902-19fb-4460-908f-7eeb6643ea5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:29.734884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 715/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:29.773451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b5891-078b-4e16-95c6-b5a0b4c2d6df None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:29.783986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b5891-078b-4e16-95c6-b5a0b4c2d6df None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:29.787458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc0990-f3a6-4ebf-82ff-06fbadd43e7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:29.788085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0990-f3a6-4ebf-82ff-06fbadd43e7b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:29.788373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0990-f3a6-4ebf-82ff-06fbadd43e7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:29.788646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc0990-f3a6-4ebf-82ff-06fbadd43e7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:29.797195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc0990-f3a6-4ebf-82ff-06fbadd43e7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:29.798616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfc0990-f3a6-4ebf-82ff-06fbadd43e7b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:29.808191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 728/1443] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:31.966590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c410e-9930-4034-bc4e-dc86aa778139 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:31.967139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c410e-9930-4034-bc4e-dc86aa778139 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:31.967139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c410e-9930-4034-bc4e-dc86aa778139 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:31.968530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 716/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:31.974306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47a1f7-0f5e-4082-82f9-b0c659213132 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:31.974733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47a1f7-0f5e-4082-82f9-b0c659213132 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:31.974931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47a1f7-0f5e-4082-82f9-b0c659213132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:31.992971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bf47a1f7-0f5e-4082-82f9-b0c659213132 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:31.994165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bf47a1f7-0f5e-4082-82f9-b0c659213132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:32.001907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 729/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:32.039966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c410e-9930-4034-bc4e-dc86aa778139 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:32.050711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c410e-9930-4034-bc4e-dc86aa778139 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:32.054238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c60e8-9151-4907-96c4-22bf4045311e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:32.054870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c60e8-9151-4907-96c4-22bf4045311e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:32.055153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c60e8-9151-4907-96c4-22bf4045311e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:32.055469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c60e8-9151-4907-96c4-22bf4045311e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:32.064125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c60e8-9151-4907-96c4-22bf4045311e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:32.065530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c60e8-9151-4907-96c4-22bf4045311e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:32.074626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 717/1446] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:34.394158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb0a91-c8f9-4eea-925a-385c366de5d6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:34.394627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb0a91-c8f9-4eea-925a-385c366de5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:34.394747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eb0a91-c8f9-4eea-925a-385c366de5d6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:34.396205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 730/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:34.401387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086fd35-576d-4ed7-b5a1-423781e62b3d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:34.401791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086fd35-576d-4ed7-b5a1-423781e62b3d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:34.401954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086fd35-576d-4ed7-b5a1-423781e62b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:34.419363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0086fd35-576d-4ed7-b5a1-423781e62b3d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:34.420552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0086fd35-576d-4ed7-b5a1-423781e62b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:34.428769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 718/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:34.467675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb0a91-c8f9-4eea-925a-385c366de5d6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:34.478960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eb0a91-c8f9-4eea-925a-385c366de5d6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:34.482496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f3eafd-2d88-4619-b704-ca0245e9226e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:34.483133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3eafd-2d88-4619-b704-ca0245e9226e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:34.483520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3eafd-2d88-4619-b704-ca0245e9226e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:34.483803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3eafd-2d88-4619-b704-ca0245e9226e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:34.492673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3eafd-2d88-4619-b704-ca0245e9226e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:34.494106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f3eafd-2d88-4619-b704-ca0245e9226e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:34.503389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 731/1449] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:36.652346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c45373-cf2a-4998-98b9-58c8beb7eadc None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:36.652913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c45373-cf2a-4998-98b9-58c8beb7eadc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:36.652913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c45373-cf2a-4998-98b9-58c8beb7eadc None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:36.654254 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 719/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:36.659540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dba83-3989-461f-937c-bdffd17b5ab6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:36.659967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dba83-3989-461f-937c-bdffd17b5ab6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:36.660099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042dba83-3989-461f-937c-bdffd17b5ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:36.678529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-042dba83-3989-461f-937c-bdffd17b5ab6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:36.679743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-042dba83-3989-461f-937c-bdffd17b5ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:36.687347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 732/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:36.725417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c45373-cf2a-4998-98b9-58c8beb7eadc None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:36.736119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c45373-cf2a-4998-98b9-58c8beb7eadc None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:36.740148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19afda4-e354-4ead-b204-4e687c37f5e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:36.740783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19afda4-e354-4ead-b204-4e687c37f5e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:36.741074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19afda4-e354-4ead-b204-4e687c37f5e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:36.741354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19afda4-e354-4ead-b204-4e687c37f5e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:36.750132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19afda4-e354-4ead-b204-4e687c37f5e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:36.751592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19afda4-e354-4ead-b204-4e687c37f5e2 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:36.761434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 720/1452] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:38.832006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7af70-3acd-4520-a9d2-13a07b224fac None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:38.832476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7af70-3acd-4520-a9d2-13a07b224fac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:38.832580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b7af70-3acd-4520-a9d2-13a07b224fac None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:38.833923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:38.839454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2411c-15bf-4feb-a2ed-db92bbe23a42 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:38.839903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2411c-15bf-4feb-a2ed-db92bbe23a42 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:38.840009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2411c-15bf-4feb-a2ed-db92bbe23a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:38.857130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1bf2411c-15bf-4feb-a2ed-db92bbe23a42 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:38.858326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1bf2411c-15bf-4feb-a2ed-db92bbe23a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:38.866598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:38.904258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7af70-3acd-4520-a9d2-13a07b224fac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:38.915084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b7af70-3acd-4520-a9d2-13a07b224fac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:38.918699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99725b-165b-4382-8875-3c17e950702c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:38.919384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99725b-165b-4382-8875-3c17e950702c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:38.919673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99725b-165b-4382-8875-3c17e950702c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:38.919952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99725b-165b-4382-8875-3c17e950702c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:38.929309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99725b-165b-4382-8875-3c17e950702c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:38.930740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd99725b-165b-4382-8875-3c17e950702c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:38.940061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 734/1455] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:41.094788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ad5ab-2610-412e-ba0e-0e9e6608f5f0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:41.095335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ad5ab-2610-412e-ba0e-0e9e6608f5f0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:41.095335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6ad5ab-2610-412e-ba0e-0e9e6608f5f0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:41.096666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:41.102497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71e0b-5159-452b-90be-9e84b4e2e266 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:41.102877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71e0b-5159-452b-90be-9e84b4e2e266 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:41.103075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71e0b-5159-452b-90be-9e84b4e2e266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:41.120375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d5c71e0b-5159-452b-90be-9e84b4e2e266 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:41.121546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d5c71e0b-5159-452b-90be-9e84b4e2e266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:41.129109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:41.166674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ad5ab-2610-412e-ba0e-0e9e6608f5f0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:41.177151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6ad5ab-2610-412e-ba0e-0e9e6608f5f0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:41.180618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17b7ae6-616e-4fc7-8bc5-0c8d4d7a31bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:41.181253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b7ae6-616e-4fc7-8bc5-0c8d4d7a31bf service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:41.181538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b7ae6-616e-4fc7-8bc5-0c8d4d7a31bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:41.181815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17b7ae6-616e-4fc7-8bc5-0c8d4d7a31bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:41.190490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b7ae6-616e-4fc7-8bc5-0c8d4d7a31bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:41.191941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17b7ae6-616e-4fc7-8bc5-0c8d4d7a31bf service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:41.201100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 723/1458] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:43.284917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd1783-efaf-42ca-9074-b5c21aae9fb1 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:43.285479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd1783-efaf-42ca-9074-b5c21aae9fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:43.285479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cd1783-efaf-42ca-9074-b5c21aae9fb1 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:43.286865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:43.292206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3efae-0f8a-42b0-87af-f5cd0c68d2ff None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:43.292693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3efae-0f8a-42b0-87af-f5cd0c68d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:43.292803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3efae-0f8a-42b0-87af-f5cd0c68d2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:43.310716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a2b3efae-0f8a-42b0-87af-f5cd0c68d2ff None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:43.311986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a2b3efae-0f8a-42b0-87af-f5cd0c68d2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:43.320488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:43.357677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd1783-efaf-42ca-9074-b5c21aae9fb1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:43.369227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cd1783-efaf-42ca-9074-b5c21aae9fb1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:43.372683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0012715a-4412-4ba6-aa94-8307484969c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:43.373309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012715a-4412-4ba6-aa94-8307484969c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:43.373590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012715a-4412-4ba6-aa94-8307484969c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:43.373863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012715a-4412-4ba6-aa94-8307484969c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:43.383215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012715a-4412-4ba6-aa94-8307484969c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:43.384740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0012715a-4412-4ba6-aa94-8307484969c2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:43.394068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 737/1461] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:45.454915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad933f2-2a1b-4fcc-ae79-b7bb215e6aa0 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:45.455498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad933f2-2a1b-4fcc-ae79-b7bb215e6aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:45.455498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad933f2-2a1b-4fcc-ae79-b7bb215e6aa0 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:45.456876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:45.462061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2010c-7632-4094-ad40-dc299ab1ddf9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:45.462738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2010c-7632-4094-ad40-dc299ab1ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:45.462884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2010c-7632-4094-ad40-dc299ab1ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:45.468868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=79 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:55:45.481887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b6c2010c-7632-4094-ad40-dc299ab1ddf9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:45.483088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b6c2010c-7632-4094-ad40-dc299ab1ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:45.491444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:45.528556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad933f2-2a1b-4fcc-ae79-b7bb215e6aa0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:45.533913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=73 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:55:45.540740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad933f2-2a1b-4fcc-ae79-b7bb215e6aa0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:45.544235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac09de-1e4f-466b-bd03-da2c7c18b48c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:45.544902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac09de-1e4f-466b-bd03-da2c7c18b48c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:45.545187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac09de-1e4f-466b-bd03-da2c7c18b48c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:45.545467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac09de-1e4f-466b-bd03-da2c7c18b48c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:45.554198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac09de-1e4f-466b-bd03-da2c7c18b48c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:45.555673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac09de-1e4f-466b-bd03-da2c7c18b48c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:45.564776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 726/1464] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:47.586262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31381f-a40a-4396-a30e-6c0b37ec517b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:47.586821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31381f-a40a-4396-a30e-6c0b37ec517b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:47.586821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a31381f-a40a-4396-a30e-6c0b37ec517b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:47.588489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:47.593726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c24fd-51c6-46f1-af21-1c71afb6584e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:47.594107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c24fd-51c6-46f1-af21-1c71afb6584e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:47.594289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c24fd-51c6-46f1-af21-1c71afb6584e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:47.612322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4c9c24fd-51c6-46f1-af21-1c71afb6584e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:47.613524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4c9c24fd-51c6-46f1-af21-1c71afb6584e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:47.621245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:47.658412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31381f-a40a-4396-a30e-6c0b37ec517b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:47.668965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a31381f-a40a-4396-a30e-6c0b37ec517b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:47.672444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbc2a58-0ae2-487b-9309-ed9e1a9286ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:47.673076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc2a58-0ae2-487b-9309-ed9e1a9286ad service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:47.673360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc2a58-0ae2-487b-9309-ed9e1a9286ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:47.673640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbc2a58-0ae2-487b-9309-ed9e1a9286ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:47.682275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc2a58-0ae2-487b-9309-ed9e1a9286ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:47.683742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbc2a58-0ae2-487b-9309-ed9e1a9286ad service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:47.693382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 740/1467] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:49.721859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0002713-f40e-4cc2-b1c0-d0c8b5731047 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:49.722403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0002713-f40e-4cc2-b1c0-d0c8b5731047 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:49.722403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0002713-f40e-4cc2-b1c0-d0c8b5731047 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:49.724364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:55:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:55:49.729786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bd904-42bd-406a-be8f-b8738d874986 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:49.730225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bd904-42bd-406a-be8f-b8738d874986 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:49.730354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bd904-42bd-406a-be8f-b8738d874986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:49.748403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-882bd904-42bd-406a-be8f-b8738d874986 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:55:49.749590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-882bd904-42bd-406a-be8f-b8738d874986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:55:49.757423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:55:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:55:49.793475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0002713-f40e-4cc2-b1c0-d0c8b5731047 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:55:49.804599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0002713-f40e-4cc2-b1c0-d0c8b5731047 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:55:49.808055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f891a85-e3e1-422d-9e53-866b9a689775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:55:49.808687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f891a85-e3e1-422d-9e53-866b9a689775 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:55:49.808968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f891a85-e3e1-422d-9e53-866b9a689775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:55:49.809243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f891a85-e3e1-422d-9e53-866b9a689775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:55:49.818040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f891a85-e3e1-422d-9e53-866b9a689775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:55:49.819496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f891a85-e3e1-422d-9e53-866b9a689775 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:55:49.829043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 729/1470] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:55:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:55:59.743272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=11 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:55:59.824772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=14 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:56:09.514895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e95ec-1f51-4269-8384-d9370fe1750a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:09.515563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e95ec-1f51-4269-8384-d9370fe1750a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:09.515563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e95ec-1f51-4269-8384-d9370fe1750a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:09.516846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 742/1471] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:09.522958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a7898-2137-4f45-8af6-80af9198bf3d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:09.523421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a7898-2137-4f45-8af6-80af9198bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:09.523523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a7898-2137-4f45-8af6-80af9198bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:09.541420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e99a7898-2137-4f45-8af6-80af9198bf3d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:09.542609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e99a7898-2137-4f45-8af6-80af9198bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:09.550186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 730/1472] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 05:56:09.596483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62e96-ca1c-4fd6-af3e-9967444facc8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:09.596791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62e96-ca1c-4fd6-af3e-9967444facc8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:09.597065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c62e96-ca1c-4fd6-af3e-9967444facc8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:09.599006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 743/1473] 10.222.0.45 () {54 vars in 962 bytes} [Mon Jun 20 05:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:09.604007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e51aa-6fd9-4a8c-8e7b-7519d4e294c4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:09.604292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e51aa-6fd9-4a8c-8e7b-7519d4e294c4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:09.604568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4e51aa-6fd9-4a8c-8e7b-7519d4e294c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:09.621941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8c4e51aa-6fd9-4a8c-8e7b-7519d4e294c4 None None] MFA Rules not processed for user `89c777401892485099d5448c506debb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:09.623236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8c4e51aa-6fd9-4a8c-8e7b-7519d4e294c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:09.629794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 731/1474] 10.222.0.45 () {58 vars in 1059 bytes} [Mon Jun 20 05:56:09 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:09.637529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62e96-ca1c-4fd6-af3e-9967444facc8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:09.648475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c62e96-ca1c-4fd6-af3e-9967444facc8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:09.651917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa7b2a9-ec7c-4dcb-9aca-9163f92bd913 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:09.652541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7b2a9-ec7c-4dcb-9aca-9163f92bd913 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:09.652821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7b2a9-ec7c-4dcb-9aca-9163f92bd913 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:09.653093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa7b2a9-ec7c-4dcb-9aca-9163f92bd913 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:09.662422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7b2a9-ec7c-4dcb-9aca-9163f92bd913 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:09.663876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa7b2a9-ec7c-4dcb-9aca-9163f92bd913 service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:09.673036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 744/1475] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:56:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 05:56:11.787800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263d5e-995d-4f9e-9ec2-8b15e5b4e65c None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:11.788363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263d5e-995d-4f9e-9ec2-8b15e5b4e65c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:11.788472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85263d5e-995d-4f9e-9ec2-8b15e5b4e65c None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:11.789764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 732/1476] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:11.794941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f270b-218f-4c48-8e31-71f788342d2a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:11.795394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f270b-218f-4c48-8e31-71f788342d2a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:11.795590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f270b-218f-4c48-8e31-71f788342d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:11.813400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-667f270b-218f-4c48-8e31-71f788342d2a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:11.814616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-667f270b-218f-4c48-8e31-71f788342d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:11.823154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 745/1477] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:11.846664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263d5e-995d-4f9e-9ec2-8b15e5b4e65c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:11.858911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85263d5e-995d-4f9e-9ec2-8b15e5b4e65c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:11.863356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:11.864518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:11.864799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:11.865073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:11.865797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:11.867580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:56:11.867580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 05:56:11.867580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 05:56:11.868740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c810701-aab6-4b35-b4d2-dbb1f9be3ced admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:11.887772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 733/1478] 10.222.0.45 () {56 vars in 1108 bytes} [Mon Jun 20 05:56:11 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:56:13.905072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a6cb-7e09-4a09-9776-15caa175d70c None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:13.905618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a6cb-7e09-4a09-9776-15caa175d70c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:13.905618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963a6cb-7e09-4a09-9776-15caa175d70c None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:13.906988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 746/1479] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:13.912155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b9667-f954-4cb3-a0c2-1f09857d6db4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:13.912586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b9667-f954-4cb3-a0c2-1f09857d6db4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:13.912727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b9667-f954-4cb3-a0c2-1f09857d6db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:13.930393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-711b9667-f954-4cb3-a0c2-1f09857d6db4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:13.931690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-711b9667-f954-4cb3-a0c2-1f09857d6db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:13.939545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 734/1480] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:13.981791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a6cb-7e09-4a09-9776-15caa175d70c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:13.996247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963a6cb-7e09-4a09-9776-15caa175d70c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:14.001568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97426b7-4950-4ba7-ad59-664d0a06aa68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:14.002200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97426b7-4950-4ba7-ad59-664d0a06aa68 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:14.002485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97426b7-4950-4ba7-ad59-664d0a06aa68 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:14.002777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97426b7-4950-4ba7-ad59-664d0a06aa68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:14.011212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97426b7-4950-4ba7-ad59-664d0a06aa68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:14.012824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97426b7-4950-4ba7-ad59-664d0a06aa68 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:14.022619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 747/1481] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:56:16.296221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af3692-e2d9-44c3-b700-7ce8a09399c3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:16.296790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af3692-e2d9-44c3-b700-7ce8a09399c3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:16.296790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76af3692-e2d9-44c3-b700-7ce8a09399c3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:16.298153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 735/1482] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:16.303870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf25f-54f1-4273-aa90-f91311039381 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:16.304277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf25f-54f1-4273-aa90-f91311039381 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:16.304434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cf25f-54f1-4273-aa90-f91311039381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:16.322233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e03cf25f-54f1-4273-aa90-f91311039381 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:16.323463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e03cf25f-54f1-4273-aa90-f91311039381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:16.331138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 748/1483] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:16.366862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af3692-e2d9-44c3-b700-7ce8a09399c3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:16.377574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76af3692-e2d9-44c3-b700-7ce8a09399c3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:16.381606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dfd1e7-dde9-4122-8980-bc142817c71d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:16.382242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd1e7-dde9-4122-8980-bc142817c71d service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:16.382533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd1e7-dde9-4122-8980-bc142817c71d service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:16.382811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dfd1e7-dde9-4122-8980-bc142817c71d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:16.391902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfd1e7-dde9-4122-8980-bc142817c71d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:16.393475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dfd1e7-dde9-4122-8980-bc142817c71d service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:16.402822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 736/1484] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:56:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:56:18.470914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f3a1-6a91-49d2-b047-6712c6652225 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:18.471468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f3a1-6a91-49d2-b047-6712c6652225 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:18.471565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75f3a1-6a91-49d2-b047-6712c6652225 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:18.473069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 749/1485] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:18.478120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0c7af-2293-445f-ac74-c471f67db727 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:18.478508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0c7af-2293-445f-ac74-c471f67db727 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:18.478679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0c7af-2293-445f-ac74-c471f67db727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:18.496076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c9a0c7af-2293-445f-ac74-c471f67db727 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:18.497420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c9a0c7af-2293-445f-ac74-c471f67db727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:18.505617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 737/1486] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:18.543637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75f3a1-6a91-49d2-b047-6712c6652225 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:18.545295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf75f3a1-6a91-49d2-b047-6712c6652225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:18.558658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75f3a1-6a91-49d2-b047-6712c6652225 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:18.562866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff64802-b5e3-4584-9ad4-a3da8b5402e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:18.563568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff64802-b5e3-4584-9ad4-a3da8b5402e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:18.563823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff64802-b5e3-4584-9ad4-a3da8b5402e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:18.564097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff64802-b5e3-4584-9ad4-a3da8b5402e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:18.573871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff64802-b5e3-4584-9ad4-a3da8b5402e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:18.575325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff64802-b5e3-4584-9ad4-a3da8b5402e4 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:18.584990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 750/1487] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:56:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:56:20.828627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ce22c-a576-45a6-8432-b17b0e0ea2cd None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:20.829082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ce22c-a576-45a6-8432-b17b0e0ea2cd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:20.829197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ce22c-a576-45a6-8432-b17b0e0ea2cd None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:20.830558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 738/1488] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:56:20.835915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0fc8-0bc2-4f96-9f68-3dcc6226c236 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:20.836346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0fc8-0bc2-4f96-9f68-3dcc6226c236 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:20.836485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc0fc8-0bc2-4f96-9f68-3dcc6226c236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:20.853962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15dc0fc8-0bc2-4f96-9f68-3dcc6226c236 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:20.855162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15dc0fc8-0bc2-4f96-9f68-3dcc6226c236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:20.862880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 751/1489] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:20.903926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ce22c-a576-45a6-8432-b17b0e0ea2cd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:20.914344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ce22c-a576-45a6-8432-b17b0e0ea2cd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:20.917829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f427d4-c254-4167-b205-fe26b095e7fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:20.918458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f427d4-c254-4167-b205-fe26b095e7fb service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:20.918740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f427d4-c254-4167-b205-fe26b095e7fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:20.919015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f427d4-c254-4167-b205-fe26b095e7fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:20.928673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f427d4-c254-4167-b205-fe26b095e7fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:20.930096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f427d4-c254-4167-b205-fe26b095e7fb service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:20.939352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 739/1490] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:56:23.193653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb5f46-c9b4-4c85-afee-45554eeeeec4 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:23.194224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb5f46-c9b4-4c85-afee-45554eeeeec4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:23.194224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eb5f46-c9b4-4c85-afee-45554eeeeec4 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:23.195664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 752/1491] 10.222.0.45 () {54 vars in 926 bytes} [Mon Jun 20 05:56:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 05:56:23.200890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1633b3-e737-4339-be26-81ad26b5eef6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:23.201320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1633b3-e737-4339-be26-81ad26b5eef6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:23.201468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1633b3-e737-4339-be26-81ad26b5eef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:23.218982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fb1633b3-e737-4339-be26-81ad26b5eef6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:56:23.220223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fb1633b3-e737-4339-be26-81ad26b5eef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:56:23.227968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 740/1492] 10.222.0.45 () {58 vars in 1023 bytes} [Mon Jun 20 05:56:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:56:23.266893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb5f46-c9b4-4c85-afee-45554eeeeec4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:56:23.277454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08eb5f46-c9b4-4c85-afee-45554eeeeec4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:56:23.280935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8c09d-0eb5-4cb7-9945-75e1114edfb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:56:23.281566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8c09d-0eb5-4cb7-9945-75e1114edfb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:56:23.281853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8c09d-0eb5-4cb7-9945-75e1114edfb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:56:23.282133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8c09d-0eb5-4cb7-9945-75e1114edfb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:56:23.290904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8c09d-0eb5-4cb7-9945-75e1114edfb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:56:23.292371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8c09d-0eb5-4cb7-9945-75e1114edfb3 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:56:23.302057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 753/1493] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:56:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 05:56:33.213984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=33 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:56:33.297472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=33 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:57:54.198680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.199275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.199275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.216918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:57:54.218099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:54.226523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 741/1494] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 05:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:57:54.235522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.236129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.236129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.254165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:57:54.255383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:54.263071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 754/1495] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 05:57:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:57:54.267321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.276775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3508e2-e32a-4ceb-b3ed-f5f306b4292b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.281120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.281759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.282042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.282322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.283040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 05:57:54.283377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.284879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.284879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 05:57:54.284879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 05:57:54.285744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.297190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 742/1496] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 05:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:57:54.305306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.314289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a4636-e24b-4c11-9218-c8223b98c271 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.318752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.319424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.319712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.319989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.320688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.322444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.322444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 05:57:54.322444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 05:57:54.323708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.358656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 755/1497] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 05:57:54 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:57:54.362846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.373235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb79b40-d756-4fd3-b88e-df23dcb42b5c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.427862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.428553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.428828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.429105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.429786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.430776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.430776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 05:57:54.430776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 05:57:54.432008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.452738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 743/1498] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 05:57:54 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 05:57:54.456711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.466933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca06dc5-1eba-4299-af50-ddaf6ffab2b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.471946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.472621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.472906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.473182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.473906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.474929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.474929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 05:57:54.474929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 05:57:54.475848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.488706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 756/1499] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 05:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:57:54.492478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.502094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a39ca26-4416-4f07-a1f1-3f2af865141d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.506641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.507320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.507607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.507885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.508600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.509592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.509592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 05:57:54.509592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 05:57:54.510467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7724e7f-4598-40fd-a229-4afba670d51e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.523298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 744/1500] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 05:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 05:57:54.527072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.537115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b45e2-6282-4267-8ba0-bd0e31ffd627 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.541552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.542418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.542704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.542983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] PATH_INFO: `/v3/projects/bcda52600b9d4d5c844ab4cd309a9785/users/c8aa059befe44f6495428a0a2ba7953f/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.557926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcda52600b9d4d5c844ab4cd309a9785, user_id=c8aa059befe44f6495428a0a2ba7953f, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.558960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.558960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 05:57:54.558960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 05:57:54.560973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71c24a-385e-4989-8280-34371ac8852e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.573435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 757/1501] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 05:57:54 2022] PUT /identity/v3/projects/bcda52600b9d4d5c844ab4cd309a9785/users/c8aa059befe44f6495428a0a2ba7953f/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:57:54.585043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb5e2d-fed8-4213-b091-f005659f7839 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.585334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb5e2d-fed8-4213-b091-f005659f7839 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.585634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb5e2d-fed8-4213-b091-f005659f7839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.611218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-46bb5e2d-fed8-4213-b091-f005659f7839 None None] MFA Rules not processed for user `c8aa059befe44f6495428a0a2ba7953f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:57:54.612457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46bb5e2d-fed8-4213-b091-f005659f7839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:54.676091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 745/1502] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 05:57:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:57:54.689044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7127dc-51aa-431d-82f4-646f99633923 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.689343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7127dc-51aa-431d-82f4-646f99633923 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.689623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7127dc-51aa-431d-82f4-646f99633923 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.691029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 758/1503] 10.222.0.45 () {54 vars in 963 bytes} [Mon Jun 20 05:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:57:54.695829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c86273-0d44-4d1b-a588-241be12babd7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.696122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c86273-0d44-4d1b-a588-241be12babd7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.696422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c86273-0d44-4d1b-a588-241be12babd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.715552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-24c86273-0d44-4d1b-a588-241be12babd7 None None] MFA Rules not processed for user `00892de83c5444c7a38b921615b2f5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:57:54.716687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-24c86273-0d44-4d1b-a588-241be12babd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:54.757967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:57:54.766201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 746/1504] 10.222.0.45 () {58 vars in 1060 bytes} [Mon Jun 20 05:57:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:57:54.774063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7127dc-51aa-431d-82f4-646f99633923 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.784599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7127dc-51aa-431d-82f4-646f99633923 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.788091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.788718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.789004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.789282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.797265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.798688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.807734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 759/1505] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:57:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:57:54.849393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb09221-c85d-48e1-b6b1-b136b9a22769 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.849683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb09221-c85d-48e1-b6b1-b136b9a22769 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.849960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb09221-c85d-48e1-b6b1-b136b9a22769 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.851243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 747/1506] 10.222.0.45 () {52 vars in 829 bytes} [Mon Jun 20 05:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:57:54.854841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:54.915225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3ec13-c233-4ef6-9171-5406e3964da1 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:54.919704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:54.920565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:54.920849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:54.921123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] PATH_INFO: `/v3/users/c8aa059befe44f6495428a0a2ba7953f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:54.922288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8aa059befe44f6495428a0a2ba7953f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:54.923341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:54.923341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 05:57:54.923341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 05:57:54.925041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:54.989766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8aa059befe44f6495428a0a2ba7953f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.001650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8aa059befe44f6495428a0a2ba7953f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.002412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8aa059befe44f6495428a0a2ba7953f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.003052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8aa059befe44f6495428a0a2ba7953f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.020534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8aa059befe44f6495428a0a2ba7953f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.024386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:57:55.037091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 760/1507] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 05:57:54 2022] DELETE /identity/v3/users/c8aa059befe44f6495428a0a2ba7953f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:57:55.041465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:55.041757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:55.042035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:55.059900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:57:55.061091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:55.117958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 748/1508] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 05:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:57:55.127367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:55.128794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:55.186112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491b91e5-3b02-4f99-9a23-529b92d076bb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:55.189839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de265220-ca08-46bc-8b6f-2c3be6d1dbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:55.190536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de265220-ca08-46bc-8b6f-2c3be6d1dbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:55.190811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de265220-ca08-46bc-8b6f-2c3be6d1dbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:55.191093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de265220-ca08-46bc-8b6f-2c3be6d1dbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:55.199843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de265220-ca08-46bc-8b6f-2c3be6d1dbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:55.201291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de265220-ca08-46bc-8b6f-2c3be6d1dbd1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:55.210518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 761/1509] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:57:55.560341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:57:55.561802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:57:55.574299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f5b29d-bc78-4e38-8810-4af46676b7db None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:57:55.584293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:57:55.585211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:57:55.585497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:57:55.585772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] PATH_INFO: `/v3/projects/bcda52600b9d4d5c844ab4cd309a9785` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:57:55.587042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcda52600b9d4d5c844ab4cd309a9785)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:57:55.588099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 05:57:55.588099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 05:57:55.588099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 05:57:55.589286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:57:55.649363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcda52600b9d4d5c844ab4cd309a9785', 'request_id': 'req-82d6a90e-a13d-4185-bedd-b0b652f8f95d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.655318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcda52600b9d4d5c844ab4cd309a9785', 'request_id': 'req-82d6a90e-a13d-4185-bedd-b0b652f8f95d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.668391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcda52600b9d4d5c844ab4cd309a9785', 'request_id': 'req-82d6a90e-a13d-4185-bedd-b0b652f8f95d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.682621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] The token cache is being invalidate because project bcda52600b9d4d5c844ab4cd309a9785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 05:57:55.682909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.683641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-82d6a90e-a13d-4185-bedd-b0b652f8f95d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 05:57:55.685187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 749/1510] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 05:57:55 2022] DELETE /identity/v3/projects/bcda52600b9d4d5c844ab4cd309a9785 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 05:58:05.206434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:58:05.679687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:59:32.501215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc53c87-e327-470b-a9f3-28492cdfb7d7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:32.501667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc53c87-e327-470b-a9f3-28492cdfb7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:32.501956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc53c87-e327-470b-a9f3-28492cdfb7d7 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:32.503174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:32.508045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fc1b8-155f-465b-b88a-4ff6e95e5fb4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:32.508489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fc1b8-155f-465b-b88a-4ff6e95e5fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:32.508622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9fc1b8-155f-465b-b88a-4ff6e95e5fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:32.526520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-da9fc1b8-155f-465b-b88a-4ff6e95e5fb4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:32.527789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-da9fc1b8-155f-465b-b88a-4ff6e95e5fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:32.584623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:32.743203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc53c87-e327-470b-a9f3-28492cdfb7d7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:32.744901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cbc53c87-e327-470b-a9f3-28492cdfb7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:32.816585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc53c87-e327-470b-a9f3-28492cdfb7d7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:32.822311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a225e9-6650-4358-b714-b1d6862f9b47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:32.822963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a225e9-6650-4358-b714-b1d6862f9b47 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:32.823265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a225e9-6650-4358-b714-b1d6862f9b47 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:32.823548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a225e9-6650-4358-b714-b1d6862f9b47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:32.832031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a225e9-6650-4358-b714-b1d6862f9b47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:32.833702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a225e9-6650-4358-b714-b1d6862f9b47 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:32.844156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 763/1513] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:33.681617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b9db-634b-4a22-aba7-fe52db01fd0f None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:33.682174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b9db-634b-4a22-aba7-fe52db01fd0f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:33.682174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93b9db-634b-4a22-aba7-fe52db01fd0f None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:33.683614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:33.689008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a327de-5efe-4369-b445-93a271114b4a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:33.689416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a327de-5efe-4369-b445-93a271114b4a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:33.689565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a327de-5efe-4369-b445-93a271114b4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:33.707354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-54a327de-5efe-4369-b445-93a271114b4a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:33.708537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-54a327de-5efe-4369-b445-93a271114b4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:33.716524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:33.868572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b9db-634b-4a22-aba7-fe52db01fd0f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:33.883428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93b9db-634b-4a22-aba7-fe52db01fd0f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:33.888692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836051c8-4d26-4da4-a8d3-8d961a4adf4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:33.889347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836051c8-4d26-4da4-a8d3-8d961a4adf4a service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:33.889636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836051c8-4d26-4da4-a8d3-8d961a4adf4a service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:33.889917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836051c8-4d26-4da4-a8d3-8d961a4adf4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:33.898519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836051c8-4d26-4da4-a8d3-8d961a4adf4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:33.900170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836051c8-4d26-4da4-a8d3-8d961a4adf4a service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:33.909648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 752/1516] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:34.504179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0136cd-0d22-4bff-b3c7-dca7404290b7 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:34.504476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0136cd-0d22-4bff-b3c7-dca7404290b7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:34.504764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0136cd-0d22-4bff-b3c7-dca7404290b7 None None] PATH_INFO: `/v3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:34.506085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 765/1517] 10.222.0.47 () {54 vars in 925 bytes} [Mon Jun 20 05:59:34 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 05:59:34.512557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a51c-59bf-4383-8315-6fb7b8e846de None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:34.512849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a51c-59bf-4383-8315-6fb7b8e846de None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:34.513132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9983a51c-59bf-4383-8315-6fb7b8e846de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:34.532023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9983a51c-59bf-4383-8315-6fb7b8e846de None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:34.533230 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9983a51c-59bf-4383-8315-6fb7b8e846de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:34.594345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 753/1518] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jun 20 05:59:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 05:59:43.334191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98e511-705c-4216-af33-935d204c706a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:43.334739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98e511-705c-4216-af33-935d204c706a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:43.334739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc98e511-705c-4216-af33-935d204c706a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:43.336067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 766/1519] 10.222.0.45 () {54 vars in 960 bytes} [Mon Jun 20 05:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:43.340899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd47515-930b-436d-9089-6810d1bde380 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:43.341565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd47515-930b-436d-9089-6810d1bde380 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:43.341733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd47515-930b-436d-9089-6810d1bde380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:43.360333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dfd47515-930b-436d-9089-6810d1bde380 None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:43.361530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dfd47515-930b-436d-9089-6810d1bde380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:43.369945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 754/1520] 10.222.0.45 () {58 vars in 1057 bytes} [Mon Jun 20 05:59:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:43.377261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98e511-705c-4216-af33-935d204c706a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:43.390747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc98e511-705c-4216-af33-935d204c706a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:43.395718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c45c19-b451-41d5-8b48-760cc856f7ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:43.396341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45c19-b451-41d5-8b48-760cc856f7ce service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:43.396638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45c19-b451-41d5-8b48-760cc856f7ce service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:43.396916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c45c19-b451-41d5-8b48-760cc856f7ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:43.405023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c45c19-b451-41d5-8b48-760cc856f7ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:43.407202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c45c19-b451-41d5-8b48-760cc856f7ce service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:43.416663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 767/1521] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:45.121514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5fb49-366d-48fa-b595-2a9dadeb4996 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:45.122045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5fb49-366d-48fa-b595-2a9dadeb4996 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:45.122045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf5fb49-366d-48fa-b595-2a9dadeb4996 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:45.123425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:45.128359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb1c11-7eac-48d9-bd69-9498650d8568 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:45.128765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb1c11-7eac-48d9-bd69-9498650d8568 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:45.128927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb1c11-7eac-48d9-bd69-9498650d8568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:45.147005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f1eb1c11-7eac-48d9-bd69-9498650d8568 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:45.148229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f1eb1c11-7eac-48d9-bd69-9498650d8568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:45.156456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:45.308706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5fb49-366d-48fa-b595-2a9dadeb4996 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:45.323811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf5fb49-366d-48fa-b595-2a9dadeb4996 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:45.329757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecfae8f-774d-4d84-bcf0-62456e250dda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:45.330396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfae8f-774d-4d84-bcf0-62456e250dda service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:45.330687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfae8f-774d-4d84-bcf0-62456e250dda service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:45.330971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecfae8f-774d-4d84-bcf0-62456e250dda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:45.339039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfae8f-774d-4d84-bcf0-62456e250dda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:45.340661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecfae8f-774d-4d84-bcf0-62456e250dda service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:45.350503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 756/1524] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:46.164869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a0626-7c1b-45c2-8100-525715bd20c6 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:46.165417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a0626-7c1b-45c2-8100-525715bd20c6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:46.165417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52a0626-7c1b-45c2-8100-525715bd20c6 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:46.166725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jun 20 05:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:46.171538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41927-aeee-4407-be50-bdb4fd3f16c1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:46.171939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41927-aeee-4407-be50-bdb4fd3f16c1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:46.172097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41927-aeee-4407-be50-bdb4fd3f16c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:46.191566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3cd41927-aeee-4407-be50-bdb4fd3f16c1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:46.192765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3cd41927-aeee-4407-be50-bdb4fd3f16c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:46.200363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jun 20 05:59:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:46.351049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a0626-7c1b-45c2-8100-525715bd20c6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:46.365734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52a0626-7c1b-45c2-8100-525715bd20c6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:46.371063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6c7a5-a72d-4732-a691-f9e1381a6907 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:46.371731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c7a5-a72d-4732-a691-f9e1381a6907 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:46.372017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c7a5-a72d-4732-a691-f9e1381a6907 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:46.372296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6c7a5-a72d-4732-a691-f9e1381a6907 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:46.380845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c7a5-a72d-4732-a691-f9e1381a6907 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:46.382443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6c7a5-a72d-4732-a691-f9e1381a6907 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:46.391674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 770/1527] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:50.429914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236134-ec97-4291-b49d-4e955bfe4eda None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:50.430502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236134-ec97-4291-b49d-4e955bfe4eda None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:50.430502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d236134-ec97-4291-b49d-4e955bfe4eda None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:50.431877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:50.437754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789fe68-4611-4811-a9b9-12a1e1db57e2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:50.438205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789fe68-4611-4811-a9b9-12a1e1db57e2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:50.438303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789fe68-4611-4811-a9b9-12a1e1db57e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:50.455874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b789fe68-4611-4811-a9b9-12a1e1db57e2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:50.457065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b789fe68-4611-4811-a9b9-12a1e1db57e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:50.464755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:50.500821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236134-ec97-4291-b49d-4e955bfe4eda None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:50.502484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0d236134-ec97-4291-b49d-4e955bfe4eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:50.558518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d236134-ec97-4291-b49d-4e955bfe4eda None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:50.562104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a9c0-829d-4df0-ac88-815793f3491f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:50.562745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a9c0-829d-4df0-ac88-815793f3491f service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:50.563032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a9c0-829d-4df0-ac88-815793f3491f service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:50.563338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a9c0-829d-4df0-ac88-815793f3491f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:50.572161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a9c0-829d-4df0-ac88-815793f3491f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:50.573571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a9c0-829d-4df0-ac88-815793f3491f service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:50.578464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:59:50.583514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 759/1530] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:52.585806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81e91-80df-4955-a559-2c820fae21d0 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:52.586316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81e91-80df-4955-a559-2c820fae21d0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:52.586436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb81e91-80df-4955-a559-2c820fae21d0 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:52.587819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:52.593061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0309f5-ef9e-48c4-9159-7098d5ec1fc0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:52.593450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0309f5-ef9e-48c4-9159-7098d5ec1fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:52.593621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0309f5-ef9e-48c4-9159-7098d5ec1fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:52.612245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ad0309f5-ef9e-48c4-9159-7098d5ec1fc0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:52.613432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ad0309f5-ef9e-48c4-9159-7098d5ec1fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:52.621400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:59:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:52.657232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81e91-80df-4955-a559-2c820fae21d0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:52.658900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2bb81e91-80df-4955-a559-2c820fae21d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:52.672389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb81e91-80df-4955-a559-2c820fae21d0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:52.676037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad5b31-b1ff-45ba-b696-a8587eb912ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:52.676893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad5b31-b1ff-45ba-b696-a8587eb912ff service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:52.677312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad5b31-b1ff-45ba-b696-a8587eb912ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:52.677724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad5b31-b1ff-45ba-b696-a8587eb912ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:52.688193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad5b31-b1ff-45ba-b696-a8587eb912ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:52.689667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cad5b31-b1ff-45ba-b696-a8587eb912ff service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:52.700485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 773/1533] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:56.203101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680799c4-f9e9-4d90-be32-21e8442ce834 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:56.203788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680799c4-f9e9-4d90-be32-21e8442ce834 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:56.203788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680799c4-f9e9-4d90-be32-21e8442ce834 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:56.205162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:59:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:56.210294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69281546-0fef-4fe1-a86a-925e8ae880bc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:56.210704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69281546-0fef-4fe1-a86a-925e8ae880bc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:56.210864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69281546-0fef-4fe1-a86a-925e8ae880bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:56.217476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=62 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 05:59:56.230493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-69281546-0fef-4fe1-a86a-925e8ae880bc None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:56.231747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-69281546-0fef-4fe1-a86a-925e8ae880bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:56.239596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:56.545428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680799c4-f9e9-4d90-be32-21e8442ce834 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:56.547113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-680799c4-f9e9-4d90-be32-21e8442ce834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:56.603259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680799c4-f9e9-4d90-be32-21e8442ce834 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:56.606761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a983b6-312f-46de-b575-a5b748df0452 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:56.607431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a983b6-312f-46de-b575-a5b748df0452 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:56.607719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a983b6-312f-46de-b575-a5b748df0452 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:56.607999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a983b6-312f-46de-b575-a5b748df0452 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:56.616886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a983b6-312f-46de-b575-a5b748df0452 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:56.618299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a983b6-312f-46de-b575-a5b748df0452 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:56.628087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 762/1536] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 05:59:58.665452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb8ace-fb05-4b3b-b81b-d511db9975f8 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:58.666027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb8ace-fb05-4b3b-b81b-d511db9975f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:58.666027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb8ace-fb05-4b3b-b81b-d511db9975f8 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:58.667465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jun 20 05:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 05:59:58.673164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e791e-3d29-4c2a-8d6b-e9690e0f3972 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:58.673556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e791e-3d29-4c2a-8d6b-e9690e0f3972 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:58.673749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e791e-3d29-4c2a-8d6b-e9690e0f3972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:58.691862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7c2e791e-3d29-4c2a-8d6b-e9690e0f3972 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 05:59:58.693058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7c2e791e-3d29-4c2a-8d6b-e9690e0f3972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:58.700709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jun 20 05:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 05:59:59.001243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb8ace-fb05-4b3b-b81b-d511db9975f8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 05:59:59.002940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a2eb8ace-fb05-4b3b-b81b-d511db9975f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 05:59:59.015796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb8ace-fb05-4b3b-b81b-d511db9975f8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 05:59:59.019330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66c1ee4-2225-48df-8a87-5e2318cdcd69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 05:59:59.019969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c1ee4-2225-48df-8a87-5e2318cdcd69 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 05:59:59.020258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c1ee4-2225-48df-8a87-5e2318cdcd69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 05:59:59.020539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66c1ee4-2225-48df-8a87-5e2318cdcd69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 05:59:59.029402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c1ee4-2225-48df-8a87-5e2318cdcd69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 05:59:59.031381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66c1ee4-2225-48df-8a87-5e2318cdcd69 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 05:59:59.041333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 776/1539] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 05:59:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:00:08.686603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=32 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:00:09.037104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:24.563750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.564959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.565473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:24.592460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:24.594089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:24.603287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 764/1540] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:24.614238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.614863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.614863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:24.644378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:24.645616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:24.660140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 777/1541] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:24.665439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:24.681080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9e260-f50f-48ca-a6da-d3a49a74b696 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:24.688216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:24.689312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.689821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.690337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:24.691398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:24.691940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:24.693787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e736f-4dba-467a-b503-690c570aed12 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:24.712128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 765/1542] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:24.721247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:24.731913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c95c06-5887-4951-b0e1-0bb6ea51a4bf None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:24.749095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:24.749789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.750189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.750471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:24.751170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:24.753130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:24.796145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 778/1543] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:24 2022] POST /identity/v3/projects => generated 383 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:24.796845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bc6e0-5085-4a2f-9097-1c468f60bd3f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.797373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bc6e0-5085-4a2f-9097-1c468f60bd3f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.797892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bc6e0-5085-4a2f-9097-1c468f60bd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:24.805749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:24.844442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e80bc6e0-5085-4a2f-9097-1c468f60bd3f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:24.846144 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e80bc6e0-5085-4a2f-9097-1c468f60bd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:24.921912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b890f17-a23a-4823-8db7-c5fb0eaaa397 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:24.932354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:24.933061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.934085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.934370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:24.935067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:24.936959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 06:01:24.936959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 06:01:24.936959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 06:01:24.938354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e7567-aa14-4d8a-acbd-ba5a6d04f6c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:24.975712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 779/1544] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:24 2022] POST /identity/v3/users => generated 292 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:01:24.977254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:24.977556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:24.977841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.013101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.013365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 766/1545] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.014765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.014969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575de17-665f-41e8-ae2f-d41123a94032 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.015229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575de17-665f-41e8-ae2f-d41123a94032 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.015539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575de17-665f-41e8-ae2f-d41123a94032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.035536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 780/1546] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.035536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.052946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4575de17-665f-41e8-ae2f-d41123a94032 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.053838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8849b1f-82d6-4818-afdb-2b84aae4460f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.054146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4575de17-665f-41e8-ae2f-d41123a94032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.059315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8515619-60cf-4d3d-b376-4f7eecac632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.059951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8515619-60cf-4d3d-b376-4f7eecac632e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.060242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8515619-60cf-4d3d-b376-4f7eecac632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.060528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8515619-60cf-4d3d-b376-4f7eecac632e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.061265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8515619-60cf-4d3d-b376-4f7eecac632e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.063042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8515619-60cf-4d3d-b376-4f7eecac632e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.063440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 767/1547] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.065025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e9b3b-7614-42cc-9f00-19d015d3faa1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.065550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e9b3b-7614-42cc-9f00-19d015d3faa1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.066072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e9b3b-7614-42cc-9f00-19d015d3faa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.078042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 781/1548] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:25.078042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca0008-d71e-442c-9c57-d1d78d42e9ae None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.078042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca0008-d71e-442c-9c57-d1d78d42e9ae None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.078042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caca0008-d71e-442c-9c57-d1d78d42e9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.090596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4d9e9b3b-7614-42cc-9f00-19d015d3faa1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.090596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4d9e9b3b-7614-42cc-9f00-19d015d3faa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.096608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-caca0008-d71e-442c-9c57-d1d78d42e9ae None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.097904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-caca0008-d71e-442c-9c57-d1d78d42e9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.100715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 768/1549] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.102006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426ad77-d64e-41b9-a66d-359abba21ace None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.102297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426ad77-d64e-41b9-a66d-359abba21ace None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.102577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426ad77-d64e-41b9-a66d-359abba21ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.107700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 782/1550] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.109134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90631d-05c1-4371-b98f-ea4d20a3cc2b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.109432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90631d-05c1-4371-b98f-ea4d20a3cc2b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.109713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90631d-05c1-4371-b98f-ea4d20a3cc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.124231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5426ad77-d64e-41b9-a66d-359abba21ace None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.125460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5426ad77-d64e-41b9-a66d-359abba21ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.128229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8e90631d-05c1-4371-b98f-ea4d20a3cc2b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.129384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8e90631d-05c1-4371-b98f-ea4d20a3cc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.133375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 769/1551] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.134586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.134886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.135170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.137321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 783/1552] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.138516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a3c05-e8a8-4b15-967a-62347775c0a4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.138805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a3c05-e8a8-4b15-967a-62347775c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.139084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a3c05-e8a8-4b15-967a-62347775c0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.153915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.155197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.157150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4b0a3c05-e8a8-4b15-967a-62347775c0a4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.158304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4b0a3c05-e8a8-4b15-967a-62347775c0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.163273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 770/1553] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.164072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.170341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 784/1554] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.170341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698505b-1a8f-4a34-8daa-ee7e98264792 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.170341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698505b-1a8f-4a34-8daa-ee7e98264792 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.170341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698505b-1a8f-4a34-8daa-ee7e98264792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.175574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1bacd3-389d-474d-b691-32013ed88a09 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.180095 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.180763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.181054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.181336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.182069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.183733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.185536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c698505b-1a8f-4a34-8daa-ee7e98264792 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.186720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c698505b-1a8f-4a34-8daa-ee7e98264792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.204253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 771/1555] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:25.204253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.205026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 785/1556] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.205026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7d8c-76d3-4299-9c26-48852b92acef None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.205026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7d8c-76d3-4299-9c26-48852b92acef None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.205026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709a7d8c-76d3-4299-9c26-48852b92acef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.209065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a16fab-15cd-4f9b-ab14-f1f91ff4c220 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.214013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.215219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.215540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.215824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.216818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.217133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.218768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.222814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-709a7d8c-76d3-4299-9c26-48852b92acef None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.224058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-709a7d8c-76d3-4299-9c26-48852b92acef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.231750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 772/1557] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.232545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.232982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 786/1558] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.236828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.236967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.237171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.248033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c256f1-6918-4a39-84c7-1a5af4ef834b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.252752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.253403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.253692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.253973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.254727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.255182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.255926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.256788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585933ad-ddac-4a0f-98aa-edc67d8e6ce3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.257119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.265655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 787/1559] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.266566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.272624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 773/1560] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.273939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.274232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.274709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.275982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a9d15-7c2c-4514-a98f-e12af9ac647c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.283162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.283833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.284127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.284403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.285124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.285432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.286451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 20 06:01:25.286451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 06:01:25.286451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 06:01:25.287356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187027ea-897d-4272-a3ba-0744551b6860 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.296136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.297370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.299278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 788/1561] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.300627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.300925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.301203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.313784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 774/1562] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.314375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.322347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.323574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.323718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a776-7b64-42dd-ab6e-0931574ac7d1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.328764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.329631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.329923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.330199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] PATH_INFO: `/v3/projects/c89583eac49a4b6286b9e1166632a76b/users/ab3c6ab7f8194d1d84a1a7716ee52cf2/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.332043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 789/1563] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.332871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.342438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2719dbfb-f1f6-454a-be5a-d95e73040228 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.344892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89583eac49a4b6286b9e1166632a76b, user_id=ab3c6ab7f8194d1d84a1a7716ee52cf2, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.345935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 06:01:25.345935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 06:01:25.345935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 06:01:25.347191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.347878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.347988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.348167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.348448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.349171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.349486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.351077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.362653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 775/1564] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:25 2022] PUT /identity/v3/projects/c89583eac49a4b6286b9e1166632a76b/users/ab3c6ab7f8194d1d84a1a7716ee52cf2/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:25.363603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.371310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 790/1565] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.371310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.381071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1becbcd-7eb6-4ef3-9640-055e39b642f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.408712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:25.417994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:25.432880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f164c2d6-fcea-4dfe-ae09-5b870f205f47 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.438017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.438680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.438969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.439271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.439388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.439966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.440074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.440257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.440563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.440978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 06:01:25.440978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 06:01:25.440978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 06:01:25.441288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.441601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.442157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.443301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.454654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 791/1566] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.455476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.465559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a761a9ef-6bbb-4424-a664-501590db01b6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.471000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.471686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.471976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.472256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.473042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 776/1567] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.473172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.473723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.475076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.502436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 792/1568] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.503582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.534860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c371c6-344d-4026-b69a-aa2179d056b1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.563882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde35a4-77c4-470d-af0e-690bd7c65c58 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.569243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.569904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.570194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.570475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.571193 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.571546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.573195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.584506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 793/1569] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.585434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.594899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8741903-421b-4e5c-997f-5e941c6f27e9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.598646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc0ab8-ba71-4abc-a3e2-c1226a246e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.599339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0ab8-ba71-4abc-a3e2-c1226a246e14 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.599476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.599625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0ab8-ba71-4abc-a3e2-c1226a246e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.599905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0ab8-ba71-4abc-a3e2-c1226a246e14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.600019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.600298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.600577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.600679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0ab8-ba71-4abc-a3e2-c1226a246e14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.601276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:25.601584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.602482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0ab8-ba71-4abc-a3e2-c1226a246e14 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.603151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.615492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 794/1570] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:25.616286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.632812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 777/1571] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.632812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.632812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.632812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.635751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5bce96-176d-4070-9442-abc7d4670fb2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.657900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] MFA Rules not processed for user `ab3c6ab7f8194d1d84a1a7716ee52cf2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:25.659132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:25.694356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.695068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.695381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.695659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.696336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.698276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.712959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:25.723838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 795/1572] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 363 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.724744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.725841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 778/1573] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:25.728076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.736422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:25.742216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6925715-80bf-4de1-b2f1-ac2e84078542 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.794325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6ca83e-dc5e-40ba-886e-571a20a11b20 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.798852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.799550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.799831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.800103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.800782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.802672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.804165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.804842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.805131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.805411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.806098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.808044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.830211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 796/1574] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.830384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 779/1575] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.831021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.831177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.842394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95697b15-0d0e-4592-b038-f2ea2c37cc4b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.892875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c658ec-0c1e-4954-84a1-a77903ad449d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.897477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.898145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.898435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.898716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.899007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.899431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.899708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.900001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.900303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.901020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.901355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.903455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.921886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 797/1576] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.922870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.933809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 780/1577] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:25.933809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:25.934084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98b3a8-566e-48f2-86a9-c9b16c112a68 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.989167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.989830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.990122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.990406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:25.991087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:25.993030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:25.993555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d848e-5001-4d65-ab46-b86e546a13c7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:25.998627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c271ba75-4403-4690-bfa9-bc768ec38395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:25.999324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ba75-4403-4690-bfa9-bc768ec38395 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:25.999615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ba75-4403-4690-bfa9-bc768ec38395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:25.999895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c271ba75-4403-4690-bfa9-bc768ec38395 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.000609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271ba75-4403-4690-bfa9-bc768ec38395 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.003498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c271ba75-4403-4690-bfa9-bc768ec38395 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.018116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:26.019609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 798/1578] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/projects => generated 391 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:26.023000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.030392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 781/1579] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:25 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:26.031767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ceb6b-70c5-43bb-9ccb-cccd7b369499 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.032060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ceb6b-70c5-43bb-9ccb-cccd7b369499 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.032341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5ceb6b-70c5-43bb-9ccb-cccd7b369499 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.033638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 782/1580] 10.222.0.45 () {54 vars in 961 bytes} [Mon Jun 20 06:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:26.034305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ceb6b-70c5-43bb-9ccb-cccd7b369499 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.035271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:26.088309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdde57-8a1e-4acf-af17-53f2ce50b81a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.093202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.093872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.094156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.094433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.095112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.097063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.100982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5ceb6b-70c5-43bb-9ccb-cccd7b369499 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.105905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.106452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.106741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.107062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.107795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.109723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.116198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 799/1581] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:26.117003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.127649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3d4959-29ea-4be9-9eaf-d15b0c5056bd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.132212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.132861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.133125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 783/1582] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:26.136185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.136185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.136956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.137058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.137058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.144031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad706f2-6acc-4c00-a81c-d84617cb2e9f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.146701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 800/1583] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.147442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.151194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.151902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.152192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.152465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.153182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.154741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.156669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d6d403-1e08-4797-8920-44f710d3d245 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.161395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.162035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.162318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.162599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.163290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.165020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 784/1584] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.165192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.165960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.176400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242cabb1-9192-4dd7-8d33-f0fefe44f319 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.181036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3bf5-1634-4d0e-895c-80459c6b290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.181680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3bf5-1634-4d0e-895c-80459c6b290b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.181964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3bf5-1634-4d0e-895c-80459c6b290b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.182241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3bf5-1634-4d0e-895c-80459c6b290b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.182912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab3bf5-1634-4d0e-895c-80459c6b290b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.185339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab3bf5-1634-4d0e-895c-80459c6b290b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.188413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 801/1585] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/users => generated 289 bytes in 38 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:26.188413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.196157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b7abb-1cf6-4cec-818f-7a131dd197db admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.202133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.202792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.203209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.203442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.204210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.205922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.206387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 785/1586] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/users => generated 296 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:26.207676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.207965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.208271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.219483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 802/1587] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.220160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.227997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:26.229213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:26.229372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c28d43-1e1a-4b2c-9486-549ca2cc9057 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.234553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.235189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.235515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.235799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.236511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.238096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.251145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 803/1588] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.251145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.259727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2723e09-5a32-4739-ba79-02636542ef7c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.264367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.264987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.265267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.265544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.266236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.267826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.278028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 804/1589] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.278870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.288470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0561a736-f7a5-4c17-9785-e0343047b180 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.293500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.294106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.294391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.294665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.295492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.297049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.299689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 786/1590] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:26.300647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.307961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 805/1591] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.309404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.314125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24290967-9af0-4cd9-99a5-927db94cbb5c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.318747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.318968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea479fb-85a7-4d44-ad48-562534527a33 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.319474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.319762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.320042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.320773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.322383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.323893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.324535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.324824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.325106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.325848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.327499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.332802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 787/1592] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.333575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.337609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 806/1593] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.338462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.343959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d15bbd-2d07-4f5c-aa08-096fb11c1fbe admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.347910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf921b4-a60f-4fc2-b574-3531e04b04be admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.348398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.349041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.349327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.349611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.350317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.351916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.353205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.354254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.354636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.355622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.356425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.358499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.362070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 788/1594] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.362070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.368715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 807/1595] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.369376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.373348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7f4c2-4115-4901-baae-0b7b4ff04f2b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.378349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.378992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.379095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6ed3a-a440-4d84-acf4-de95136217dd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.379305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.379594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.380302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.381859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.383654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.384275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.384559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.384833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.385527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.387137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.392384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 789/1596] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.396259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.399895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 808/1597] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.399895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.406800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c070b463-1316-433c-9797-c1f8ee51b52d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.408390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a3d5f-fc8e-4565-82c7-54e2133fac83 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.411484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.412342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.412627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.412906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] PATH_INFO: `/v3/projects/9fd70cbaefa049e299c3aabd01549d65/users/70323e3276ca4aaa87f55c5fb38837aa/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.414108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.414735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.415026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.415340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.428397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fd70cbaefa049e299c3aabd01549d65, user_id=70323e3276ca4aaa87f55c5fb38837aa, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.428563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:26.431136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.443484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 809/1598] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/9fd70cbaefa049e299c3aabd01549d65/users/70323e3276ca4aaa87f55c5fb38837aa/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.444240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.474200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:26.478266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.480623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.489694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 790/1599] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:26.490491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.500434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f258e540-5792-4354-ba3a-9b9e43b78002 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.505200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.505841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.505968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604ab0-13c3-41e0-bb5b-cf528b362af1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.506131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.506414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.507132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.508719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.510852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.512446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.512730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.513005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] PATH_INFO: `/v3/projects/b9f1c2212dfa4413a0293a6fb2b8d940/users/42cbf1f18ba94c04b5d68acc47b2adf4/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.519698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 791/1600] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.520380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.531277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26643ca7-f87f-4f65-ae5f-159b2ac52272 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.532360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9f1c2212dfa4413a0293a6fb2b8d940, user_id=42cbf1f18ba94c04b5d68acc47b2adf4, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.535004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.536091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.536793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.537093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.537373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.538090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.539710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.549191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 810/1601] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/b9f1c2212dfa4413a0293a6fb2b8d940/users/42cbf1f18ba94c04b5d68acc47b2adf4/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.552981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 792/1602] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:26.553135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.553242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.561233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998a8df-fcb0-499a-8faf-0771c03b1fbd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.619893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500efeb-87e2-4411-8473-2d4c07708528 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.622853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8662fc55-c451-4927-b567-2927a4cfc276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.624055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662fc55-c451-4927-b567-2927a4cfc276 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.624406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662fc55-c451-4927-b567-2927a4cfc276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.624745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8662fc55-c451-4927-b567-2927a4cfc276 admin admin] PATH_INFO: `/v3/projects/92fb6ddd92bd45cb805d9dd2419650cd/users/c9ae97d080f847e1bdd16d76354c2654/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.625431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.626393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.626748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.627095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] PATH_INFO: `/v3/projects/643a3ccceba44a79931f512607d868a7/users/b8757543ed624ee3be3df54c9f02c9e1/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.641913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662fc55-c451-4927-b567-2927a4cfc276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92fb6ddd92bd45cb805d9dd2419650cd, user_id=c9ae97d080f847e1bdd16d76354c2654, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.642107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643a3ccceba44a79931f512607d868a7, user_id=b8757543ed624ee3be3df54c9f02c9e1, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.644764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.645043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8662fc55-c451-4927-b567-2927a4cfc276 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.658791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 811/1603] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/643a3ccceba44a79931f512607d868a7/users/b8757543ed624ee3be3df54c9f02c9e1/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.659358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.666951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 793/1604] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/92fb6ddd92bd45cb805d9dd2419650cd/users/c9ae97d080f847e1bdd16d76354c2654/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.669142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.669581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.669893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.689905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:26.691421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:26.734790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e95d8-7f83-465a-b48d-827b8afc6dd6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.757876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 794/1605] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:26.758846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.768320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d4723-d92d-4f86-897d-c8ab9fbf45bd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.773014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29c91-ff6f-4d85-a34f-d3a1ba121dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.773894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29c91-ff6f-4d85-a34f-d3a1ba121dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.774178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29c91-ff6f-4d85-a34f-d3a1ba121dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.774456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29c91-ff6f-4d85-a34f-d3a1ba121dd5 admin admin] PATH_INFO: `/v3/projects/cf7ee2aaec40481da0190b662acfde8d/users/adab7140fc804689af945257511cfded/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.778259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:26.790261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c29c91-ff6f-4d85-a34f-d3a1ba121dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf7ee2aaec40481da0190b662acfde8d, user_id=adab7140fc804689af945257511cfded, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.793631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c29c91-ff6f-4d85-a34f-d3a1ba121dd5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.795425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.795849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:26.796341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.796624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.797072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] PATH_INFO: `/v3/projects/8899c92c0c9b4b69af1d9aa2d0d8ee03/users/dfbd42cb9a0d4c1ab17f74c967067f08/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.808192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 795/1606] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/cf7ee2aaec40481da0190b662acfde8d/users/adab7140fc804689af945257511cfded/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.809518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993c18f-dc8c-4cb2-b460-c3824b1f064e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.809808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993c18f-dc8c-4cb2-b460-c3824b1f064e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.810088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1993c18f-dc8c-4cb2-b460-c3824b1f064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.814561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8899c92c0c9b4b69af1d9aa2d0d8ee03, user_id=dfbd42cb9a0d4c1ab17f74c967067f08, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.818894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.827204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1993c18f-dc8c-4cb2-b460-c3824b1f064e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:26.828719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1993c18f-dc8c-4cb2-b460-c3824b1f064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:26.830686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 812/1607] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/8899c92c0c9b4b69af1d9aa2d0d8ee03/users/dfbd42cb9a0d4c1ab17f74c967067f08/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.831456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:26.892421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 796/1608] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:26.893579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a1c6a2-8f77-49d6-8a85-296f77215192 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:26.893917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c685af-aa23-445a-b0e7-13a9ae475de5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.894267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c685af-aa23-445a-b0e7-13a9ae475de5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.894629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c685af-aa23-445a-b0e7-13a9ae475de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.898239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee69c0c-a066-4d0b-b2bb-f70e7f2055ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:26.899131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee69c0c-a066-4d0b-b2bb-f70e7f2055ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.899446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee69c0c-a066-4d0b-b2bb-f70e7f2055ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.899727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee69c0c-a066-4d0b-b2bb-f70e7f2055ea admin admin] PATH_INFO: `/v3/projects/e339d5680b35466b8df59e6770716c4f/users/ca882ae475a24e7f84124a1ab3e374f6/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.914636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee69c0c-a066-4d0b-b2bb-f70e7f2055ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=e339d5680b35466b8df59e6770716c4f, user_id=ca882ae475a24e7f84124a1ab3e374f6, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:26.917315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee69c0c-a066-4d0b-b2bb-f70e7f2055ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:26.921400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b3c685af-aa23-445a-b0e7-13a9ae475de5 None None] MFA Rules not processed for user `b8757543ed624ee3be3df54c9f02c9e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:26.922833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b3c685af-aa23-445a-b0e7-13a9ae475de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:26.930744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 813/1609] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:26 2022] PUT /identity/v3/projects/e339d5680b35466b8df59e6770716c4f/users/ca882ae475a24e7f84124a1ab3e374f6/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:26.932209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dab3ed-f4f0-44d7-9c9a-90c86bb885d9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.932968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dab3ed-f4f0-44d7-9c9a-90c86bb885d9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.932968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dab3ed-f4f0-44d7-9c9a-90c86bb885d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.958906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-48dab3ed-f4f0-44d7-9c9a-90c86bb885d9 None None] MFA Rules not processed for user `c9ae97d080f847e1bdd16d76354c2654`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:26.960246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48dab3ed-f4f0-44d7-9c9a-90c86bb885d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:26.989677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 797/1610] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:26.991143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb9c91-da08-45e7-ab58-29fd3d85b211 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:26.991463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb9c91-da08-45e7-ab58-29fd3d85b211 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:26.991742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb9c91-da08-45e7-ab58-29fd3d85b211 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:26.993140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 798/1611] 10.222.0.45 () {54 vars in 912 bytes} [Mon Jun 20 06:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:26.993827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb9c91-da08-45e7-ab58-29fd3d85b211 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.031606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 814/1612] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:27.033148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd08e-6e74-4a91-b73a-89f97173e73a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.033438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd08e-6e74-4a91-b73a-89f97173e73a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.034704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342fd08e-6e74-4a91-b73a-89f97173e73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.052488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-342fd08e-6e74-4a91-b73a-89f97173e73a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:27.053675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-342fd08e-6e74-4a91-b73a-89f97173e73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:27.054512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb9c91-da08-45e7-ab58-29fd3d85b211 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.058063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.059429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.059714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.059991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.086017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:27.098941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:27.113624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 815/1613] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:27.115841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f0272-9f9f-4646-a70e-aa41d9cd77dd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.115841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f0272-9f9f-4646-a70e-aa41d9cd77dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.115841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f0272-9f9f-4646-a70e-aa41d9cd77dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.132588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.134020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.135029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9b1f0272-9f9f-4646-a70e-aa41d9cd77dd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:27.136308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9b1f0272-9f9f-4646-a70e-aa41d9cd77dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:27.143740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 799/1614] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.144061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 816/1615] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:27.144503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.145171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639ccab-c2f8-4e7b-af9c-53b1e614914d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.145459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639ccab-c2f8-4e7b-af9c-53b1e614914d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.145737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639ccab-c2f8-4e7b-af9c-53b1e614914d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.152250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6c6b3b-0f76-4ecc-bec3-4b3139c721cb service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.156128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6fcfe-a272-4d76-aa11-b80a8e448086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.156768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6fcfe-a272-4d76-aa11-b80a8e448086 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.157056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6fcfe-a272-4d76-aa11-b80a8e448086 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.157360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6fcfe-a272-4d76-aa11-b80a8e448086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.163652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4639ccab-c2f8-4e7b-af9c-53b1e614914d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:27.164851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4639ccab-c2f8-4e7b-af9c-53b1e614914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:27.168074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6fcfe-a272-4d76-aa11-b80a8e448086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.169522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d6fcfe-a272-4d76-aa11-b80a8e448086 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.174234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 817/1616] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:27.176966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.177708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.178568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.179903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 800/1617] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.181222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a4a21-fbe6-41be-8b10-55f31040c87b None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.181514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a4a21-fbe6-41be-8b10-55f31040c87b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.181794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a4a21-fbe6-41be-8b10-55f31040c87b None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.183093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 801/1618] 10.222.0.45 () {54 vars in 961 bytes} [Mon Jun 20 06:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:27.184368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a4a21-fbe6-41be-8b10-55f31040c87b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.199001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a4a21-fbe6-41be-8b10-55f31040c87b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.205482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.206219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.206572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.206968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.207711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.210281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.213940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:27.216579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:27.247470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 802/1619] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:27.248389 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.323989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 818/1620] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:27.324963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.335361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9422989-e1f4-48e2-bd7a-5dae7a3eb905 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.336727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68669b36-8fe8-4e45-9d17-41e9e92c83bd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.342789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40566c30-462c-4352-8087-3c56af6dcfbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.342892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.342892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.343047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566c30-462c-4352-8087-3c56af6dcfbc service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.343121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.343202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566c30-462c-4352-8087-3c56af6dcfbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.343309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.343388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40566c30-462c-4352-8087-3c56af6dcfbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.431331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.431553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.443772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40566c30-462c-4352-8087-3c56af6dcfbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.447385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40566c30-462c-4352-8087-3c56af6dcfbc service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.449028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 819/1621] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.451916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.465114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 803/1622] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.467429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.467872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.468217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c90d6d-cb5a-4705-b285-6a94a9259721 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.468538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.473329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.474049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.474452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.474770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.489118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.490959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:27.491136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.492771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:27.508135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 820/1623] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.509027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.521098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ae444-20a4-44f2-80d3-9ba4e2be69ef service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.526562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.527290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.527709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.528099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.528797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.530857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.552191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:27.556134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 821/1624] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:27.557609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.588165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 804/1625] 10.222.0.45 () {58 vars in 1058 bytes} [Mon Jun 20 06:01:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:27.589487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.600547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:27.604288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a2a21a-4b98-47ba-9ddb-de3207bdc2a6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.609565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.610838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.610838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.611425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.612180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.614322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.631693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 805/1626] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:27.631693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.645481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d3135-fb4c-4dca-9e39-331050cf3b17 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.646416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89007231-263b-41a8-b2c1-f46087d2cbb3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.650830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.651988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.652655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.652942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.653141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.653267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.653484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.653772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.676265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.679214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.696295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 822/1627] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.697376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.710979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de451ba-0fcf-4822-bd2e-9c3f389441d3 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.719206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.720511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.720846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.721031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.721141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] PATH_INFO: `/v3/projects/23e3d6863fb6424c84d53cc061868814/users/705008fc04454af8b7237c10a8b25c1d/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.722734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.733785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 806/1628] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:27.743105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e3d6863fb6424c84d53cc061868814, user_id=705008fc04454af8b7237c10a8b25c1d, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.746216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.775291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 823/1629] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:27 2022] PUT /identity/v3/projects/23e3d6863fb6424c84d53cc061868814/users/705008fc04454af8b7237c10a8b25c1d/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:27.777773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.794003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187bcc95-fd74-4463-96da-3d132e9b11d6 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.931165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df491074-26f1-4f09-90b0-edd169ef2316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.932167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491074-26f1-4f09-90b0-edd169ef2316 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.932672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491074-26f1-4f09-90b0-edd169ef2316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.933201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df491074-26f1-4f09-90b0-edd169ef2316 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.934081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df491074-26f1-4f09-90b0-edd169ef2316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:27.936229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df491074-26f1-4f09-90b0-edd169ef2316 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:27.953276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 807/1630] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:27.957586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:27.975568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9a5d32-87d8-42a6-b798-b48a45dd52e6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:27.988872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e9b1d-c36c-486c-8812-a5e2de36e836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:27.990715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e9b1d-c36c-486c-8812-a5e2de36e836 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:27.991293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e9b1d-c36c-486c-8812-a5e2de36e836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:27.991293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e9b1d-c36c-486c-8812-a5e2de36e836 admin admin] PATH_INFO: `/v3/projects/23e3d6863fb6424c84d53cc061868814/users/705008fc04454af8b7237c10a8b25c1d/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:27.995676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e9b1d-c36c-486c-8812-a5e2de36e836 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e3d6863fb6424c84d53cc061868814, user_id=705008fc04454af8b7237c10a8b25c1d, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:28.000689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e9b1d-c36c-486c-8812-a5e2de36e836 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:28.031830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 824/1631] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:27 2022] PUT /identity/v3/projects/23e3d6863fb6424c84d53cc061868814/users/705008fc04454af8b7237c10a8b25c1d/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:28.043430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:28.044020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:28.045125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:28.109304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] MFA Rules not processed for user `705008fc04454af8b7237c10a8b25c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:28.117880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:28.188735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1b9c9-d854-4e1d-95dc-8d86c5094a1d None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:28.189328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1b9c9-d854-4e1d-95dc-8d86c5094a1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:28.189756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd1b9c9-d854-4e1d-95dc-8d86c5094a1d None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:28.191590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 825/1632] 10.222.0.45 () {54 vars in 912 bytes} [Mon Jun 20 06:01:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:01:28.203443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:28.203984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:28.205717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:28.247232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:28.250229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:28.254420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:28.276105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 808/1633] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:28 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:28.290086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:28.424548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8626c57f-56fe-4870-82c9-39967a8fca5a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:28.431803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 826/1634] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:01:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:28.432614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:28.433917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:28.434369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:28.434750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:28.459665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:28.462259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:28.482621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 809/1635] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:28 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:28.531049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:28.546590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1f4a0-0441-4e0c-923f-ba8cee2a7fdc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:28.556219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2006e-cbe3-44ea-956c-55da150412a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:28.561397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2006e-cbe3-44ea-956c-55da150412a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:28.561866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2006e-cbe3-44ea-956c-55da150412a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:28.562313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2006e-cbe3-44ea-956c-55da150412a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:28.574563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2006e-cbe3-44ea-956c-55da150412a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:28.576268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2006e-cbe3-44ea-956c-55da150412a7 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:28.605840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 827/1636] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:28.994369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:29.006284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5553af7a-2b63-4392-abf9-033c99794cd9 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:29.013177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:29.014088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:29.014479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:29.015186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:29.120413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:29.123066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:29.155547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 810/1637] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:28 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:29.343235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:29.343235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:29.343817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:29.421382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] MFA Rules not processed for user `42cbf1f18ba94c04b5d68acc47b2adf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:29.428133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:29.552661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:29.568724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 828/1638] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:29.572640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:29.678235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2562ed61-ab2a-42a7-94db-cccd9c1fde88 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:29.684848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:29.686520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:29.686990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:29.687516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:29.689199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:29.693748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:29.730062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 811/1639] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:29 2022] POST /identity/v3/projects => generated 403 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:29.733859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:29.751596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a0a47-19f9-4349-a097-618b7a2fe538 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:29.864893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:29.865604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:29.865894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:29.866171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:29.866883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:29.869036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:29.897612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 829/1640] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:29 2022] POST /identity/v3/users => generated 308 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:29.901517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:29.914327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fda7dd-e10d-45e4-9183-53446c88178d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:29.923115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:29.924712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:29.925005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:29.925291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:29.926039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:29.927847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:29.945938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 812/1641] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:01:29.952227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:29.965989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5edfd-7db6-41b8-aad8-419cba54fa1d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:29.972617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:29.974051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:29.974411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:29.974699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] PATH_INFO: `/v3/projects/1bc3261d40764419a4461a01e1342aec/users/1f472a46bd44473b892842472f7f1e93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:30.007963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc3261d40764419a4461a01e1342aec, user_id=1f472a46bd44473b892842472f7f1e93, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:30.010915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:30.042288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 830/1642] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:29 2022] PUT /identity/v3/projects/1bc3261d40764419a4461a01e1342aec/users/1f472a46bd44473b892842472f7f1e93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:30.046714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:30.061469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:30.071291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ee987-0080-4d2d-ae55-4fb0524b5635 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:30.172713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf587156-832c-44e8-8cb5-1a20e989a0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:30.173695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587156-832c-44e8-8cb5-1a20e989a0a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:30.174200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587156-832c-44e8-8cb5-1a20e989a0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:30.174703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587156-832c-44e8-8cb5-1a20e989a0a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:30.176063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf587156-832c-44e8-8cb5-1a20e989a0a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:30.178781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf587156-832c-44e8-8cb5-1a20e989a0a8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:30.203507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:30.205741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 813/1643] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:30.219017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98ed4-7d0a-4f5c-aca9-815c00fc0ff6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:30.224630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:30.225640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:30.225924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:30.226201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] PATH_INFO: `/v3/projects/1bc3261d40764419a4461a01e1342aec/users/1f472a46bd44473b892842472f7f1e93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:30.229316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bc3261d40764419a4461a01e1342aec, user_id=1f472a46bd44473b892842472f7f1e93, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:30.233327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:30.256920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 831/1644] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:30 2022] PUT /identity/v3/projects/1bc3261d40764419a4461a01e1342aec/users/1f472a46bd44473b892842472f7f1e93/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:33.140807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e6de3-5f31-4285-bd1a-7e4f571993a8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:33.141405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e6de3-5f31-4285-bd1a-7e4f571993a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:33.142493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e6de3-5f31-4285-bd1a-7e4f571993a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:33.206374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c81e6de3-5f31-4285-bd1a-7e4f571993a8 None None] MFA Rules not processed for user `1f472a46bd44473b892842472f7f1e93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:33.209853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c81e6de3-5f31-4285-bd1a-7e4f571993a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:33.424998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 814/1645] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:33 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:35.243137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:35.482868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f0550a-9a5a-4452-b4af-46c9f8055e6c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:35.499243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:35.500779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:35.501071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:35.501352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] PATH_INFO: `/v3/users/b8757543ed624ee3be3df54c9f02c9e1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:35.505432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8757543ed624ee3be3df54c9f02c9e1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:35.510812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:35.662735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8757543ed624ee3be3df54c9f02c9e1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:35.666713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:35.689943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8757543ed624ee3be3df54c9f02c9e1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:35.690838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8757543ed624ee3be3df54c9f02c9e1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:35.691559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8757543ed624ee3be3df54c9f02c9e1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:35.732401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2fc27dbf-be77-43a6-ba27-8f74ee1f5ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8757543ed624ee3be3df54c9f02c9e1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:35.771014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 832/1646] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:35 2022] DELETE /identity/v3/users/b8757543ed624ee3be3df54c9f02c9e1 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:35.772976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:35.773494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:35.773566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:35.820950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:35.824456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:35.992630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321c386-9cbf-4fcf-b750-1633554b9a1a None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:35.992943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321c386-9cbf-4fcf-b750-1633554b9a1a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:35.993383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8321c386-9cbf-4fcf-b750-1633554b9a1a None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:35.995016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 833/1647] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:01:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:36.055561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde437-a526-4c15-bb58-4e81af10ecba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:36.055561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde437-a526-4c15-bb58-4e81af10ecba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:36.055561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde437-a526-4c15-bb58-4e81af10ecba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:36.077890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 815/1648] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:36.093812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7dbde437-a526-4c15-bb58-4e81af10ecba None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:36.098661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7dbde437-a526-4c15-bb58-4e81af10ecba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:36.149293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:36.152473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:36.258292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:36.343338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bec5c0-9aab-4300-a105-7cfa136c3b76 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:36.352191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402593d-621d-4070-bd00-0642780ad383 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:36.353136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402593d-621d-4070-bd00-0642780ad383 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:36.353604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402593d-621d-4070-bd00-0642780ad383 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:36.353902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402593d-621d-4070-bd00-0642780ad383 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:36.373736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402593d-621d-4070-bd00-0642780ad383 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:36.375611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402593d-621d-4070-bd00-0642780ad383 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:36.390604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 816/1649] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:36.399975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 834/1650] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:01:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:36.723754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8322315-ac15-466a-a687-5c68c1b9386a None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:36.724774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8322315-ac15-466a-a687-5c68c1b9386a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:36.725103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8322315-ac15-466a-a687-5c68c1b9386a None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:36.726595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 817/1651] 10.222.0.45 () {54 vars in 960 bytes} [Mon Jun 20 06:01:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:36.736540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:36.737082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:36.737526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:36.771241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:36.772537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:36.788758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 835/1652] 10.222.0.45 () {58 vars in 1057 bytes} [Mon Jun 20 06:01:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:36.799122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8322315-ac15-466a-a687-5c68c1b9386a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:36.825633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8322315-ac15-466a-a687-5c68c1b9386a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:36.840448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ab698-bb19-40bd-b1bc-84f829fbc640 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:36.842792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ab698-bb19-40bd-b1bc-84f829fbc640 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:36.843279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ab698-bb19-40bd-b1bc-84f829fbc640 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:36.843699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ab698-bb19-40bd-b1bc-84f829fbc640 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:36.871159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ab698-bb19-40bd-b1bc-84f829fbc640 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:36.873128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ab698-bb19-40bd-b1bc-84f829fbc640 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:36.921611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 818/1653] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:37.374963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:37.374963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:37.403083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfce27c-7ace-4f49-b390-e742fe379a9e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:37.427971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:37.429117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:37.429400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:37.429674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] PATH_INFO: `/v3/users/ab3c6ab7f8194d1d84a1a7716ee52cf2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:37.431449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3c6ab7f8194d1d84a1a7716ee52cf2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:37.437274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:37.600026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3c6ab7f8194d1d84a1a7716ee52cf2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:37.619038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3c6ab7f8194d1d84a1a7716ee52cf2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:37.620834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3c6ab7f8194d1d84a1a7716ee52cf2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:37.627792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3c6ab7f8194d1d84a1a7716ee52cf2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:37.661877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3c6ab7f8194d1d84a1a7716ee52cf2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:37.692548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 836/1654] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:37 2022] DELETE /identity/v3/users/ab3c6ab7f8194d1d84a1a7716ee52cf2 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:37.697536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:37.697959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:37.698411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:37.724463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:37.725958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:37.917840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 819/1655] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:37.932570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:37.934616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:38.003654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:38.056230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c875eb-e65a-440f-a675-09a1c4397297 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:38.064795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2e47e-813c-4823-bdaa-3cfd9b60aa34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:38.065830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2e47e-813c-4823-bdaa-3cfd9b60aa34 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:38.066228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2e47e-813c-4823-bdaa-3cfd9b60aa34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:38.066728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2e47e-813c-4823-bdaa-3cfd9b60aa34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:38.093774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2e47e-813c-4823-bdaa-3cfd9b60aa34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:38.096608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2e47e-813c-4823-bdaa-3cfd9b60aa34 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:38.118549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 837/1656] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:38.843261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:38.848110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:38.871896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9978b5-343a-4688-bd22-2850207aeef5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:38.890494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:38.892184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:38.892483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:38.892768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] PATH_INFO: `/v3/projects/c89583eac49a4b6286b9e1166632a76b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:38.896876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89583eac49a4b6286b9e1166632a76b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:38.901606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:39.012356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89583eac49a4b6286b9e1166632a76b', 'request_id': 'req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:39.023742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89583eac49a4b6286b9e1166632a76b', 'request_id': 'req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:39.038215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89583eac49a4b6286b9e1166632a76b', 'request_id': 'req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:39.061747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] The token cache is being invalidate because project c89583eac49a4b6286b9e1166632a76b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:39.062043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:39.063383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6737b88c-1a14-4ee1-a972-9daa2b94ed4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:39.070806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.071328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.071429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.074493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 820/1657] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:38 2022] DELETE /identity/v3/projects/c89583eac49a4b6286b9e1166632a76b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:39.095099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:39.096624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:39.286994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 838/1658] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:39.296035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.296442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.296971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.322166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:39.325489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:39.353299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 821/1659] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:39.354832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:39.371924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9e890-62c1-4eaa-b562-ac55ceb9176d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:39.397365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:39.398077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.398374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.398658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.399440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:39.400702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:39.402589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:39.436359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 839/1660] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:39.439445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:39.454261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47076d49-087d-412b-8792-7c52b34376f2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:39.481793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:39.483883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.484424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.484975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.486533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:39.490140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:39.565125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 822/1661] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:39 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:39.569344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:39.589532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4d5703-6a19-465b-9a64-b723da86c2ba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:39.770805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:39.771694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.777368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.777905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.778812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:39.781808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:39.839848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 840/1662] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:39 2022] POST /identity/v3/users => generated 293 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:39.842325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:39.858279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96e2d1f-86a6-4022-8ec6-fef8334f419d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:39.866183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:39.868599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a075f-e0ca-4ebc-b4d4-04912cf3550f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:39.870515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:39.871569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a075f-e0ca-4ebc-b4d4-04912cf3550f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.875147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a075f-e0ca-4ebc-b4d4-04912cf3550f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.875853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a075f-e0ca-4ebc-b4d4-04912cf3550f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.876906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a075f-e0ca-4ebc-b4d4-04912cf3550f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:39.880920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a075f-e0ca-4ebc-b4d4-04912cf3550f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:39.891157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5d6c3a-2488-4b3d-9ab7-9170f4a9fb52 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:39.902805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:39.909767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:39.910854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.911213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.911594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] PATH_INFO: `/v3/projects/643a3ccceba44a79931f512607d868a7` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.913291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643a3ccceba44a79931f512607d868a7)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:39.914415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: WARNING py.warnings [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 06:01:39.914415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: warnings.warn(msg) Jun 20 06:01:39.914415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: Jun 20 06:01:39.918315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:39.925438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 823/1663] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:39.932102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec02c80-44bf-4ea6-be5d-69db5e6ec84e None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.932828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec02c80-44bf-4ea6-be5d-69db5e6ec84e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.933437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec02c80-44bf-4ea6-be5d-69db5e6ec84e None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.935292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 824/1664] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:01:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:01:39.936208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec02c80-44bf-4ea6-be5d-69db5e6ec84e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:39.981398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec02c80-44bf-4ea6-be5d-69db5e6ec84e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:39.995638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a335d9-51bc-4592-b3bb-c48c8b9dab85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:39.996333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a335d9-51bc-4592-b3bb-c48c8b9dab85 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:39.996619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a335d9-51bc-4592-b3bb-c48c8b9dab85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:39.997279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a335d9-51bc-4592-b3bb-c48c8b9dab85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:39.998488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a335d9-51bc-4592-b3bb-c48c8b9dab85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:40.004449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a335d9-51bc-4592-b3bb-c48c8b9dab85 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:40.051967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:40.056522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 825/1665] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:40.059293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f363f44-2d87-4108-90c2-524ec7499af9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:40.059839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f363f44-2d87-4108-90c2-524ec7499af9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:40.060329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f363f44-2d87-4108-90c2-524ec7499af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:40.074265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643a3ccceba44a79931f512607d868a7', 'request_id': 'req-dcf71043-e230-447f-b1bc-78c7b985e095'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.095867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643a3ccceba44a79931f512607d868a7', 'request_id': 'req-dcf71043-e230-447f-b1bc-78c7b985e095'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.144687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0f363f44-2d87-4108-90c2-524ec7499af9 None None] MFA Rules not processed for user `ca882ae475a24e7f84124a1ab3e374f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:40.148299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0f363f44-2d87-4108-90c2-524ec7499af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:40.152384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643a3ccceba44a79931f512607d868a7', 'request_id': 'req-dcf71043-e230-447f-b1bc-78c7b985e095'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.196081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] The token cache is being invalidate because project 643a3ccceba44a79931f512607d868a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:40.196397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.199448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.203736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 841/1666] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:39 2022] DELETE /identity/v3/projects/643a3ccceba44a79931f512607d868a7 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:40.211479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:40.213615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:40.384048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 826/1667] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:40.385515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:40.393921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:40.394303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:40.438901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf71043-e230-447f-b1bc-78c7b985e095 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:40.454709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] MFA Rules not processed for user `e93c507849c04762bf311c0c3108bab3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:40.455982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:40.464374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db5c9e8-b019-4367-824d-e90dfa4f8f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:40.465308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5c9e8-b019-4367-824d-e90dfa4f8f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:40.465599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5c9e8-b019-4367-824d-e90dfa4f8f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:40.465883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db5c9e8-b019-4367-824d-e90dfa4f8f0e admin admin] PATH_INFO: `/v3/projects/6567c382a79b459286c245c83978fd46/users/e5ee3346445246e1a2f88eac09222dca/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:40.518063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5c9e8-b019-4367-824d-e90dfa4f8f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6567c382a79b459286c245c83978fd46, user_id=e5ee3346445246e1a2f88eac09222dca, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:40.525052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db5c9e8-b019-4367-824d-e90dfa4f8f0e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:40.573161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 842/1668] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:40 2022] PUT /identity/v3/projects/6567c382a79b459286c245c83978fd46/users/e5ee3346445246e1a2f88eac09222dca/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:40.576507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a9bd7-f341-4634-8092-809533ecb879 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:40.578084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a9bd7-f341-4634-8092-809533ecb879 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:40.581355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5a9bd7-f341-4634-8092-809533ecb879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:40.629269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0b5a9bd7-f341-4634-8092-809533ecb879 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:40.630511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0b5a9bd7-f341-4634-8092-809533ecb879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:40.632680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 827/1669] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:01:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:40.633586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:40.640550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:40.662080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29cb149-da69-44fe-aa42-f3857e82a0ea None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:40.755571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:40.758184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:40.759617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:40.759767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] PATH_INFO: `/v3/users/42cbf1f18ba94c04b5d68acc47b2adf4` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:40.762933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=42cbf1f18ba94c04b5d68acc47b2adf4)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:40.765680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: WARNING py.warnings [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 20 06:01:40.765680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: warnings.warn(msg) Jun 20 06:01:40.765680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: Jun 20 06:01:40.770068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:40.778312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 843/1670] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:40.780037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bda428-6413-497d-810f-f7bc71317723 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:40.780554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bda428-6413-497d-810f-f7bc71317723 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:40.780979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bda428-6413-497d-810f-f7bc71317723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:40.831843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-00bda428-6413-497d-810f-f7bc71317723 None None] MFA Rules not processed for user `ca882ae475a24e7f84124a1ab3e374f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:40.833512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00bda428-6413-497d-810f-f7bc71317723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:40.856197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:40.937932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42cbf1f18ba94c04b5d68acc47b2adf4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.964730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42cbf1f18ba94c04b5d68acc47b2adf4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.966792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42cbf1f18ba94c04b5d68acc47b2adf4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:40.968650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42cbf1f18ba94c04b5d68acc47b2adf4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:41.001161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:41.021453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42cbf1f18ba94c04b5d68acc47b2adf4'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:41.038887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 844/1671] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:41.042244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.042545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.042830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.066049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 828/1672] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:40 2022] DELETE /identity/v3/users/42cbf1f18ba94c04b5d68acc47b2adf4 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:41.067284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:41.073850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.192480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] MFA Rules not processed for user `e5ee3346445246e1a2f88eac09222dca`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:41.198706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.270831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4374360-dc40-4924-94da-c1a917fcb9fd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:41.282757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:41.283876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.284330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.285347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.292192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.317411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:41.319464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019eee7d-8c1a-4f3a-af55-0c537ee6d74c service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:41.353372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 829/1673] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:41.365876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.366083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.366380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.405058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 845/1674] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:41.405976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:41.409336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.416467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:41.417770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.587314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 830/1675] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:41.591725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:41.592378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.605535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932630de-c809-4059-a5d4-65fec6b13cca None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:41.618211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:41.618437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.618602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.618867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.619917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd772c1-396a-44fc-af40-1b61d342e81a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:41.623338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:41.629448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:41.630132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.630414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.630691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.631417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:41.638161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:41.691194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 831/1676] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:41 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:41.692287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:41.701191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:41.785868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:41.791403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:41.824607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 846/1677] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:41.825605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:41.863222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f866bba-d368-49be-b6ce-ee489c10ccf2 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:41.881744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:41.882496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a9eec1-90af-4f28-b4d7-a1c4e51b9757 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:41.882614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.882869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.883155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.883991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:41.887991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:41.907992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:41.908689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.908982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.909261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.910001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:41.910307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:41.911932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc87fbc-b847-4c47-a0c5-c8018454e3d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:41.950512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 832/1678] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:41.957667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:41.958547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:41.962962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:41.985003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 847/1679] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:41 2022] POST /identity/v3/users => generated 299 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:41.985920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.040031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5717b6-1b2c-4896-8cf6-d3ba8d1b8e6d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.063641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.065105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.065586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.066368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.068322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] MFA Rules not processed for user `dfbd42cb9a0d4c1ab17f74c967067f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:42.069579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:42.069861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.072954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.081644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:42.167700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 848/1680] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:41 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:42.168670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.282004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 833/1681] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:42.283102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.313292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87dd565-c3d2-44fe-a8e4-281c3e542042 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.322013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251952-d3d5-4124-ac30-c7bafed4ca6a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.326440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.327420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.328002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.328603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.329932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.334701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.336410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6551f2-b91a-44f5-8562-683a5bcd8006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.338448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6551f2-b91a-44f5-8562-683a5bcd8006 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.338851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6551f2-b91a-44f5-8562-683a5bcd8006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.343824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6551f2-b91a-44f5-8562-683a5bcd8006 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.344643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6551f2-b91a-44f5-8562-683a5bcd8006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.347803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6551f2-b91a-44f5-8562-683a5bcd8006 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.368246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 849/1682] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:42.369825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.371082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.371643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.387996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 834/1683] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:42 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:42.387996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.435359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df3f4e1-5377-4add-830c-ea4f747f2bf5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.444442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.445162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.445447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.445724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] PATH_INFO: `/v3/projects/2dd281cbd1044b73bf8f2041f2c569c6/users/97c56b2aaf7048b7a7115a06d9a5c7ec/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.449157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] MFA Rules not processed for user `dfbd42cb9a0d4c1ab17f74c967067f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:42.450390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:42.481721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd281cbd1044b73bf8f2041f2c569c6, user_id=97c56b2aaf7048b7a7115a06d9a5c7ec, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.486407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.515760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 835/1684] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:42 2022] PUT /identity/v3/projects/2dd281cbd1044b73bf8f2041f2c569c6/users/97c56b2aaf7048b7a7115a06d9a5c7ec/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:42.516828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.596589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 850/1685] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:42.597616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.602664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:42.641654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bde44-44eb-4979-9e82-c9682a5d35a7 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.657768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:42.696501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bdc18d-76aa-451c-9243-b85fb0428532 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.710313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.710987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.711995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.712192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.714969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.719417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.746771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 836/1686] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:42.749485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.777446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19232f-7cd7-49fd-b614-925c897dfff6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:42.797103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.797844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.798128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.798409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.799138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.801179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:42.802421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:42.802807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:42.803547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] PATH_INFO: `/v3/projects/b9f1c2212dfa4413a0293a6fb2b8d940` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:42.808053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9f1c2212dfa4413a0293a6fb2b8d940)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:42.809615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.811534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:42.849577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 837/1687] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:42.849577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.855154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:42.884437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:42.923748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9f1c2212dfa4413a0293a6fb2b8d940', 'request_id': 'req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:42.935668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9f1c2212dfa4413a0293a6fb2b8d940', 'request_id': 'req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:42.944050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9f1c2212dfa4413a0293a6fb2b8d940', 'request_id': 'req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:42.964154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] The token cache is being invalidate because project b9f1c2212dfa4413a0293a6fb2b8d940 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:42.965140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:42.965999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:42.968898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 851/1688] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:42 2022] DELETE /identity/v3/projects/b9f1c2212dfa4413a0293a6fb2b8d940 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:42.970226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:42.972395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:43.062323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac074e54-b960-416a-9b74-c6e9cf08a6a3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:43.064118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12db28f2-8b8b-4b23-afa9-ec0102bbad0d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:43.070329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:43.071219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.071640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.072021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.073339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:43.076219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:43.105986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 852/1689] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:43.105986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:43.108725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:43.140606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492d257-5eea-4ccf-bf3d-de9cc3c888c4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:43.158034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:43.158257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.159295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.159538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] PATH_INFO: `/v3/projects/2dd281cbd1044b73bf8f2041f2c569c6/users/97c56b2aaf7048b7a7115a06d9a5c7ec/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.163299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2dd281cbd1044b73bf8f2041f2c569c6, user_id=97c56b2aaf7048b7a7115a06d9a5c7ec, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:43.165978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:43.197406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 853/1690] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:43 2022] PUT /identity/v3/projects/2dd281cbd1044b73bf8f2041f2c569c6/users/97c56b2aaf7048b7a7115a06d9a5c7ec/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:43.198172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:43.209379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:43.248716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:43.250194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.250682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.251190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.254152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:43.379933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57940189-4f20-4f33-8b31-18f81bf689e6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:43.389405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:43.390325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.390609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.390887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] PATH_INFO: `/v3/users/1f472a46bd44473b892842472f7f1e93` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.397933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f472a46bd44473b892842472f7f1e93)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:43.401905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:43.413935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:43.415710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:43.452538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 838/1691] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:43.453693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:43.472608 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:43.481402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390df13-8858-4893-bc60-0e35738d69ec service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:43.506010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f445050-47c5-489d-abec-2f127dacade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:43.508059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f445050-47c5-489d-abec-2f127dacade5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.508668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f445050-47c5-489d-abec-2f127dacade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.509320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f445050-47c5-489d-abec-2f127dacade5 admin admin] PATH_INFO: `/v3/projects/c6c227432e8e4bf5b4003ce612b8822a/users/c141d894856d42ea8495cc47ee158bcf/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.526802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:43.538762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f472a46bd44473b892842472f7f1e93'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:43.552387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f472a46bd44473b892842472f7f1e93'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:43.553017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f445050-47c5-489d-abec-2f127dacade5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c227432e8e4bf5b4003ce612b8822a, user_id=c141d894856d42ea8495cc47ee158bcf, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:43.553346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f472a46bd44473b892842472f7f1e93'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:43.554055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f472a46bd44473b892842472f7f1e93'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:43.556664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f445050-47c5-489d-abec-2f127dacade5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:43.586526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f6366425-e660-4400-a9c3-cff5972b4a31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f472a46bd44473b892842472f7f1e93'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:43.615340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 839/1692] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:43 2022] PUT /identity/v3/projects/c6c227432e8e4bf5b4003ce612b8822a/users/c141d894856d42ea8495cc47ee158bcf/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:43.617393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0712291-44fb-4627-99e8-ebc3ee12ac5e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.617790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0712291-44fb-4627-99e8-ebc3ee12ac5e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.618706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0712291-44fb-4627-99e8-ebc3ee12ac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.626017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 854/1693] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:43 2022] DELETE /identity/v3/users/1f472a46bd44473b892842472f7f1e93 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:43.631213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaea35-8165-4ca4-9ce4-a2a697830a33 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.634352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaea35-8165-4ca4-9ce4-a2a697830a33 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.634791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eaea35-8165-4ca4-9ce4-a2a697830a33 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.660905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 855/1694] 10.222.0.45 () {54 vars in 912 bytes} [Mon Jun 20 06:01:43 2022] GET /identity => generated 270 bytes in 34 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:43.666168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090de7-e0fb-439f-bfea-c0a1c9d65546 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.669216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090de7-e0fb-439f-bfea-c0a1c9d65546 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.669697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08090de7-e0fb-439f-bfea-c0a1c9d65546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.696450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e0712291-44fb-4627-99e8-ebc3ee12ac5e None None] MFA Rules not processed for user `97c56b2aaf7048b7a7115a06d9a5c7ec`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:43.697530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e0712291-44fb-4627-99e8-ebc3ee12ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:43.756722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-08090de7-e0fb-439f-bfea-c0a1c9d65546 None None] MFA Rules not processed for user `70323e3276ca4aaa87f55c5fb38837aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:43.764485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08090de7-e0fb-439f-bfea-c0a1c9d65546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:43.898946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 840/1695] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:43.900463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.903095 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.903420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.936100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 856/1696] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:43.943778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:43.943778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:43.943778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:43.997633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:43.999132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.012531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] MFA Rules not processed for user `c141d894856d42ea8495cc47ee158bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:44.015683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.124398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 857/1697] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:01:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:44.125062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:44.128963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.198452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:44.214964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0651aec-2215-4021-8686-4312e6a16a14 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:44.221532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:44.221890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:44.222261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:44.222444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:44.242238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:44.247769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:44.270502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 858/1698] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:44.271491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:44.273157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.373445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca5b9a0-c82c-4b4f-9ff4-b23790d8cd26 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:44.378908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:44.380134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:44.380695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:44.381031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] PATH_INFO: `/v3/users/c9ae97d080f847e1bdd16d76354c2654` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:44.382468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9ae97d080f847e1bdd16d76354c2654)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:44.384860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:44.497097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9ae97d080f847e1bdd16d76354c2654'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.505900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9ae97d080f847e1bdd16d76354c2654'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.511048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9ae97d080f847e1bdd16d76354c2654'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.512345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9ae97d080f847e1bdd16d76354c2654'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.529445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9ae97d080f847e1bdd16d76354c2654'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.550977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 859/1699] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:44 2022] DELETE /identity/v3/users/c9ae97d080f847e1bdd16d76354c2654 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:44.552266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:44.554253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.555639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 841/1700] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 656 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:44.556732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:44.560265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.589605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a960ae9e-ee79-4f62-8429-367955e9465b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:44.636214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2ed6-2f3f-4de4-95de-b696486d417b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:44.640872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:44.641608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:44.641967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:44.642580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:44.647076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.669931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:44.674099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:44.690337 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 860/1701] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:44.694625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:44.700912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:44.716653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:44.727019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:44.728093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:44.728984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:44.729396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:44.729618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] PATH_INFO: `/v3/users/e5ee3346445246e1a2f88eac09222dca` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:44.740275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5ee3346445246e1a2f88eac09222dca)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:44.742265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:44.885051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5ee3346445246e1a2f88eac09222dca'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.889532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8143bef3-3e2d-482d-8a79-a71f2f102eb2 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:44.905442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:44.906941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5ee3346445246e1a2f88eac09222dca'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.907719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:44.909106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:44.909852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] PATH_INFO: `/v3/projects/1bc3261d40764419a4461a01e1342aec` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:44.910114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5ee3346445246e1a2f88eac09222dca'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.911443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5ee3346445246e1a2f88eac09222dca'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:44.912820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bc3261d40764419a4461a01e1342aec)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:44.931264 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:44.966971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-48dfe7ec-8b62-4bd0-bf30-e96558bfd3c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5ee3346445246e1a2f88eac09222dca'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:45.020351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 842/1702] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:44 2022] DELETE /identity/v3/users/e5ee3346445246e1a2f88eac09222dca => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:45.026735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99402-4538-411e-b5a1-000abd1eb197 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.028732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99402-4538-411e-b5a1-000abd1eb197 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.029628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c99402-4538-411e-b5a1-000abd1eb197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.093136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-40c99402-4538-411e-b5a1-000abd1eb197 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:45.104176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40c99402-4538-411e-b5a1-000abd1eb197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.116235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bc3261d40764419a4461a01e1342aec', 'request_id': 'req-c870b0cc-a791-4b42-b216-9ecec7f80bd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:45.145256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bc3261d40764419a4461a01e1342aec', 'request_id': 'req-c870b0cc-a791-4b42-b216-9ecec7f80bd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:45.170897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bc3261d40764419a4461a01e1342aec', 'request_id': 'req-c870b0cc-a791-4b42-b216-9ecec7f80bd0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:45.204351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] The token cache is being invalidate because project 1bc3261d40764419a4461a01e1342aec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:45.204637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:45.205347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c870b0cc-a791-4b42-b216-9ecec7f80bd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:45.208150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 861/1703] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:44 2022] DELETE /identity/v3/projects/1bc3261d40764419a4461a01e1342aec => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:45.211550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0556d0-4547-46ae-8236-f5b19c41c75b None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.212772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0556d0-4547-46ae-8236-f5b19c41c75b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.213059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0556d0-4547-46ae-8236-f5b19c41c75b None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.215768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 862/1704] 10.222.0.45 () {54 vars in 912 bytes} [Mon Jun 20 06:01:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:45.220577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3ac3-6724-42a9-bcac-df9b37368aad None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.221481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3ac3-6724-42a9-bcac-df9b37368aad None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.221771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789c3ac3-6724-42a9-bcac-df9b37368aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.269702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-789c3ac3-6724-42a9-bcac-df9b37368aad None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:45.271991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 843/1705] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:45.274463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-789c3ac3-6724-42a9-bcac-df9b37368aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.275006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.275377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.275725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.295414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 863/1706] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:45.316414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.316704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.316985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.342547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:45.350725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.379764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 844/1707] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:45.379979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:45.379979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.380129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.380478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.524364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 864/1708] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:01:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:45.525308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.538314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2c1b7-8572-4c84-b6d4-1724ce98b1ea None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:45.544957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e91a1-f4af-46f0-bdc2-d72a6dbab036 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:45.547595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9060b-55a7-4e2c-ae97-e8eea8ccfa07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:45.548314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9060b-55a7-4e2c-ae97-e8eea8ccfa07 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.548661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9060b-55a7-4e2c-ae97-e8eea8ccfa07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.548942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9060b-55a7-4e2c-ae97-e8eea8ccfa07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.557422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:45.560498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.561364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9060b-55a7-4e2c-ae97-e8eea8ccfa07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:45.562109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.562519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.562880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9060b-55a7-4e2c-ae97-e8eea8ccfa07 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:45.607038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:45.608974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:45.610995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 845/1709] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:45.613382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.613634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.613915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.628237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 865/1710] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:45.633986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.651630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb6603-a419-4fe8-b832-3837df5fcb2c service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:45.654979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:45.656236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.661498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:45.662167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.662454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.662789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.679450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 846/1711] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:45.683363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.701243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:45.705490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:45.725246 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:45.730265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 866/1712] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:45.732373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7374c56-eeb7-4fc5-a055-021b803406b4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:45.732668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.738471 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.745520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:45.746676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.747000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.747470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.748341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:45.749265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:45.752221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:45.771434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 847/1713] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:45.779983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:45.791475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.837100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91707163-20bb-4cee-b4d1-8808c70a11f3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:45.850695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:45.851489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:45.851779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:45.854113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:45.856372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:45.859622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:45.913098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 848/1714] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:45 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:45.913971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:45.918417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:45.933187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73a1014-3628-4476-9775-681048fd4063 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.075858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3d5b4-ad3f-4e8b-893c-8d04ba507781 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.094098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.095439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.096675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.097671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.106705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:46.124297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.127510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.128975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.129273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.132900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:46.136436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.138590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.164071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 849/1715] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:46.165833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:46.233514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.236023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c017c04-97d7-4ac4-ba9b-cf034b17fd92 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.263008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 867/1716] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:46.268756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.269113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.269651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.299726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc74dc4-dade-4fe3-8ce9-fd72fc1e64d1 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.310635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.312904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.313220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.313506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.316483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.321628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.358575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] MFA Rules not processed for user `adab7140fc804689af945257511cfded`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:46.362935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:46.389079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 850/1717] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:46 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:46.394129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:46.448160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed68a3e-747b-4315-b035-3b13b2bf6f22 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.468821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.469561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.469867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.470140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.470898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.473252 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.486171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:46.502197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 851/1718] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:46.503204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:46.550787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 868/1719] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:46.550787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:46.557481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da24365b-8753-4e03-9fa2-758343ab2c95 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.578076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.578707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853846d8-a592-4bdb-a32c-820b8dec3037 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.579804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.580475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.581044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.585134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.585636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.585878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.586330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.587188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.588867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:46.589671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.606011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 869/1720] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:46.606011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:46.606619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:46.627510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f17e4f-d74a-4b7b-b744-2810bd626eb7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.640305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:46.641763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:46.642059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:46.643030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] PATH_INFO: `/v3/projects/92fb6ddd92bd45cb805d9dd2419650cd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:46.644475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92fb6ddd92bd45cb805d9dd2419650cd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.646529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.711367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:46.803168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:46.808397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:46.836548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92fb6ddd92bd45cb805d9dd2419650cd', 'request_id': 'req-503c9784-e942-478a-ad4b-fcd654aa3c66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:46.879484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 852/1721] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:46.882234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:46.910186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92fb6ddd92bd45cb805d9dd2419650cd', 'request_id': 'req-503c9784-e942-478a-ad4b-fcd654aa3c66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:46.948765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92fb6ddd92bd45cb805d9dd2419650cd', 'request_id': 'req-503c9784-e942-478a-ad4b-fcd654aa3c66'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:46.973391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a69fa-86e6-4b96-b7de-438cbf967613 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:46.996805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] The token cache is being invalidate because project 92fb6ddd92bd45cb805d9dd2419650cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:46.997113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.000603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.011223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 870/1722] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:46 2022] DELETE /identity/v3/projects/92fb6ddd92bd45cb805d9dd2419650cd => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:47.017772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:47.022689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:47.140962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c9784-e942-478a-ad4b-fcd654aa3c66 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:47.147239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:47.148179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:47.148469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:47.148746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] PATH_INFO: `/v3/users/705008fc04454af8b7237c10a8b25c1d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:47.150443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] RBAC: Authorizing `identity:delete_user(user_id=705008fc04454af8b7237c10a8b25c1d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:47.153253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:47.169004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2386fc-75f3-4a2c-b92d-ac2e3d3b2b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:47.170388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2386fc-75f3-4a2c-b92d-ac2e3d3b2b66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:47.170684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2386fc-75f3-4a2c-b92d-ac2e3d3b2b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:47.171163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2386fc-75f3-4a2c-b92d-ac2e3d3b2b66 admin admin] PATH_INFO: `/v3/projects/bcb94fc880224954ba8aa9fb88d647c2/users/c2fafe5b46404a97b306f7eb85256988/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:47.248869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2386fc-75f3-4a2c-b92d-ac2e3d3b2b66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb94fc880224954ba8aa9fb88d647c2, user_id=c2fafe5b46404a97b306f7eb85256988, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:47.257543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2386fc-75f3-4a2c-b92d-ac2e3d3b2b66 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:47.314425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 853/1723] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:46 2022] PUT /identity/v3/projects/bcb94fc880224954ba8aa9fb88d647c2/users/c2fafe5b46404a97b306f7eb85256988/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:47.325488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:47.325782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:47.326064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:47.327629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '705008fc04454af8b7237c10a8b25c1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.338031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:47.344218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '705008fc04454af8b7237c10a8b25c1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.345004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '705008fc04454af8b7237c10a8b25c1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.345639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '705008fc04454af8b7237c10a8b25c1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.405120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4555b90e-2808-4416-9694-e9747c6ef368 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '705008fc04454af8b7237c10a8b25c1d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:47.432309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 871/1724] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:47 2022] DELETE /identity/v3/users/705008fc04454af8b7237c10a8b25c1d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:47.433587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] MFA Rules not processed for user `c2fafe5b46404a97b306f7eb85256988`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:47.433819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:47.434122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:47.434452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:47.439321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:47.473854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] MFA Rules not processed for user `ca882ae475a24e7f84124a1ab3e374f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:47.475294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:47.612460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 872/1725] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:47.613860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:47.615473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:47.637183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 854/1726] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:47.644271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:47.648284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:47.834975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:47.842460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfa740-deaf-4e33-a0c1-9c1405aafa07 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:47.856349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:47.858330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:47.859079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:47.859079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:47.865097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:47.912620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80326a-da36-4b37-9cca-207fe6c9248f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:47.939538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:47.941059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:47.941434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:47.941727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:47.942433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:47.946690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.039765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 855/1727] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:47 2022] POST /identity/v3/projects => generated 365 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:48.048999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:48.119771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:48.121409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.224759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 873/1728] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 612 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:48.227582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:48.245922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:48.298169 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710dc6e-ac79-4f9f-a0d2-3bbc8a105df1 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:48.313305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d11aa4-7763-4428-9116-acfe17c458a5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:48.330634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:48.333050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:48.333688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:48.333976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:48.334253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:48.334571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:48.334882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:48.334990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:48.335166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] PATH_INFO: `/v3/projects/6567c382a79b459286c245c83978fd46` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:48.336903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.343950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6567c382a79b459286c245c83978fd46)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:48.347429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.411396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 856/1729] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:48 2022] POST /identity/v3/users => generated 289 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:48.413211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:48.417856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:48.453892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b18c2-62cf-43e5-9764-f2986746232d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:48.472302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:48.473103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:48.473279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:48.473563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:48.474423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:48.479046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.520491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6567c382a79b459286c245c83978fd46', 'request_id': 'req-c79f3222-f969-464e-9b2b-bd011cd14eaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.522184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 857/1730] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:48.527311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:48.531317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6567c382a79b459286c245c83978fd46', 'request_id': 'req-c79f3222-f969-464e-9b2b-bd011cd14eaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.538960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6567c382a79b459286c245c83978fd46', 'request_id': 'req-c79f3222-f969-464e-9b2b-bd011cd14eaf'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.545785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b027291-83de-4d4d-a8a5-ba77f9c8ac4d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:48.558140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] The token cache is being invalidate because project 6567c382a79b459286c245c83978fd46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:48.558561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.559440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.562232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 874/1731] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:48 2022] DELETE /identity/v3/projects/6567c382a79b459286c245c83978fd46 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:48.565972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:48.568199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:48.688793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:48.692849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:48.693249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:48.693793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] PATH_INFO: `/v3/projects/79ef203fdee84e0ab1466f8a90d8820b/users/b442ef2b6806452286e8400fbf5c67ce/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:48.733308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ef203fdee84e0ab1466f8a90d8820b, user_id=b442ef2b6806452286e8400fbf5c67ce, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:48.738254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f3222-f969-464e-9b2b-bd011cd14eaf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:48.739317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.749475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:48.750441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:48.750734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:48.751013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] PATH_INFO: `/v3/users/97c56b2aaf7048b7a7115a06d9a5c7ec` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:48.753409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97c56b2aaf7048b7a7115a06d9a5c7ec)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:48.755443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:48.782238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 858/1732] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:48 2022] PUT /identity/v3/projects/79ef203fdee84e0ab1466f8a90d8820b/users/b442ef2b6806452286e8400fbf5c67ce/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:48.783240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:48.802414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:48.892103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97c56b2aaf7048b7a7115a06d9a5c7ec'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.924223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:48.933094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97c56b2aaf7048b7a7115a06d9a5c7ec'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.934150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97c56b2aaf7048b7a7115a06d9a5c7ec'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.936380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97c56b2aaf7048b7a7115a06d9a5c7ec'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.974746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1bacbe-0d78-4eb4-a05d-8f961b3052af admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:48.989455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97c56b2aaf7048b7a7115a06d9a5c7ec'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:48.995932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:48.999231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:49.000326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:49.001203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:49.004145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:49.009640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:49.044407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 875/1733] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:48 2022] DELETE /identity/v3/users/97c56b2aaf7048b7a7115a06d9a5c7ec => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:49.048936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 859/1734] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:49.052774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:49.055789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:49.111998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:49.133952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:49.136812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:49.166306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0677c2-9c00-48fd-b3b8-8c08605e1a18 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:49.273519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48a9149-3696-437d-95c3-9b8274a9df36 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:49.292755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635321f9-ed66-45db-9a1c-348db8269b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:49.293804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635321f9-ed66-45db-9a1c-348db8269b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:49.294159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635321f9-ed66-45db-9a1c-348db8269b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:49.294542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635321f9-ed66-45db-9a1c-348db8269b73 admin admin] PATH_INFO: `/v3/projects/79ef203fdee84e0ab1466f8a90d8820b/users/b442ef2b6806452286e8400fbf5c67ce/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:49.298695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635321f9-ed66-45db-9a1c-348db8269b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79ef203fdee84e0ab1466f8a90d8820b, user_id=b442ef2b6806452286e8400fbf5c67ce, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:49.304518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635321f9-ed66-45db-9a1c-348db8269b73 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:49.333759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:49.343578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:49.353739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 876/1735] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:49 2022] PUT /identity/v3/projects/79ef203fdee84e0ab1466f8a90d8820b/users/b442ef2b6806452286e8400fbf5c67ce/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:49.355696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:49.356196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] PATH_INFO: `/v3/users/c141d894856d42ea8495cc47ee158bcf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:49.356348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:49.356701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:49.357136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:49.358461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c141d894856d42ea8495cc47ee158bcf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:49.360951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:49.440835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] MFA Rules not processed for user `b442ef2b6806452286e8400fbf5c67ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:49.444236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:49.471568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c141d894856d42ea8495cc47ee158bcf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:49.483004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c141d894856d42ea8495cc47ee158bcf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:49.483835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c141d894856d42ea8495cc47ee158bcf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:49.484554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c141d894856d42ea8495cc47ee158bcf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:49.512427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-92bdbba5-c95f-4ab8-b485-004f54964b96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c141d894856d42ea8495cc47ee158bcf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:49.540735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 860/1736] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:49 2022] DELETE /identity/v3/users/c141d894856d42ea8495cc47ee158bcf => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:49.545085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:49.546093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:49.546366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:49.585298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:49.586566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:49.673698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 877/1737] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:49 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:49.686371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:49.688578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:49.732586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:49.802729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 861/1738] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:49.882598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ea42a-b2ba-4c6c-9b9a-d44f9a8ef850 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:49.952078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:49.953677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:50.058337 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe438c9-bbea-4550-a423-61e713cd86bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:50.059021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe438c9-bbea-4550-a423-61e713cd86bb service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.059333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe438c9-bbea-4550-a423-61e713cd86bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.059616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe438c9-bbea-4550-a423-61e713cd86bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.072462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:50.164359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee457-ea43-4a36-957a-b061d1ce08b3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:50.170693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:50.171847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.172486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.173103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.190152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:50.194065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:50.224081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe438c9-bbea-4550-a423-61e713cd86bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:50.225621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe438c9-bbea-4550-a423-61e713cd86bb service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:50.235210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 862/1739] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:50.245273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 878/1740] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:49 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:50.317077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:50.318806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:50.346300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a7f75-5fa6-490f-9189-48323e45d914 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:50.368140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:50.369067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.369751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.369751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] PATH_INFO: `/v3/projects/23e3d6863fb6424c84d53cc061868814` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.370981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e3d6863fb6424c84d53cc061868814)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:50.373134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:50.473462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88701f80-a766-4725-a2d4-9e68649a42a1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.473758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88701f80-a766-4725-a2d4-9e68649a42a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.474135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88701f80-a766-4725-a2d4-9e68649a42a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.480044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e3d6863fb6424c84d53cc061868814', 'request_id': 'req-87741ef6-71a6-4d10-8c95-2d08af85480e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:50.499467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e3d6863fb6424c84d53cc061868814', 'request_id': 'req-87741ef6-71a6-4d10-8c95-2d08af85480e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:50.514405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e3d6863fb6424c84d53cc061868814', 'request_id': 'req-87741ef6-71a6-4d10-8c95-2d08af85480e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:50.517204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-88701f80-a766-4725-a2d4-9e68649a42a1 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:50.518449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-88701f80-a766-4725-a2d4-9e68649a42a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:50.545746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] The token cache is being invalidate because project 23e3d6863fb6424c84d53cc061868814 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:50.546011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:50.546532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87741ef6-71a6-4d10-8c95-2d08af85480e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:50.553767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 863/1741] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:50 2022] DELETE /identity/v3/projects/23e3d6863fb6424c84d53cc061868814 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:50.555183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.557236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.558098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.594276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:50.597318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:50.666416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 879/1742] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:01:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:50.764233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.764528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.764815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.767488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 864/1743] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:50.828086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:50.829605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:50.852906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 880/1744] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:50.859957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:50.878479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bde5e1-ce94-4699-97f5-46730e5cfcaa None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:50.913926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:50.914733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:50.915035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:50.915386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:50.916281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:50.917156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:50.921339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:50.964251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 865/1745] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:50.966791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:50.987592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581ebb-8d5d-4e1f-b1bf-bd9b85a26e01 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:51.001163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:51.027045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.027478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.027775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.032114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:51.043960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:51.076724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:51.078834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:51.086794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:51.145514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 881/1746] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:50 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:51.156249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:51.247927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9897-0de2-47f4-8882-e41e88fad5c8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:51.276072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332c832-b453-4e19-8151-669daa152334 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:51.283075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:51.299881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.310248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.311839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.313942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:51.319238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:51.361333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 882/1747] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:51 2022] POST /identity/v3/users => generated 294 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:51.362266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:51.372244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:51.392512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:51.393331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.393729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.394058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.410684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734dbe93-fa9c-4d12-958a-86e77856f3b3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:51.423723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:51.427996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.429060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.429391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] PATH_INFO: `/v3/users/c2fafe5b46404a97b306f7eb85256988` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.437459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2fafe5b46404a97b306f7eb85256988)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:51.446697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:51.451783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:51.477116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:51.478705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:51.490045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 866/1748] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:01:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 06:01:51.491645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:51.512629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f310d6f8-9fea-4e5d-bcaf-bcf9534fa2c4 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:51.519237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:51.519978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.520274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.520618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.522334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:51.524689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:51.548473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 867/1749] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:51.548473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:51.548767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:51.587032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaea122-d0e1-4a1f-bf7d-8cb7f70130bc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:51.596814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:51.597718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.598010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.598296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] PATH_INFO: `/v3/projects/2dd281cbd1044b73bf8f2041f2c569c6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.599601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2dd281cbd1044b73bf8f2041f2c569c6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:51.601761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:51.660068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2fafe5b46404a97b306f7eb85256988'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.699727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:51.700061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2fafe5b46404a97b306f7eb85256988'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.700061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2fafe5b46404a97b306f7eb85256988'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.701097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2fafe5b46404a97b306f7eb85256988'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.722987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2dd281cbd1044b73bf8f2041f2c569c6', 'request_id': 'req-065af96f-01ea-40cb-8117-cf99c1798a51'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.737534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2dd281cbd1044b73bf8f2041f2c569c6', 'request_id': 'req-065af96f-01ea-40cb-8117-cf99c1798a51'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.755312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-54419c03-ec20-44ab-be37-a5568157f9ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2fafe5b46404a97b306f7eb85256988'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.756541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2dd281cbd1044b73bf8f2041f2c569c6', 'request_id': 'req-065af96f-01ea-40cb-8117-cf99c1798a51'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.790683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] The token cache is being invalidate because project 2dd281cbd1044b73bf8f2041f2c569c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:51.793436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.796152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:51.798351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 868/1750] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:51 2022] DELETE /identity/v3/projects/2dd281cbd1044b73bf8f2041f2c569c6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:51.799191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:51.802712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:51.809693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 883/1751] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:51 2022] DELETE /identity/v3/users/c2fafe5b46404a97b306f7eb85256988 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:51.812964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71c016-360d-4b3a-991a-43d959387f45 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:51.813270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71c016-360d-4b3a-991a-43d959387f45 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:51.813539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db71c016-360d-4b3a-991a-43d959387f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:51.926671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-db71c016-360d-4b3a-991a-43d959387f45 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:51.930806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db71c016-360d-4b3a-991a-43d959387f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:52.014782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065af96f-01ea-40cb-8117-cf99c1798a51 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:52.188408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed0c52-e339-471c-9e4f-1521154f4881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:52.189381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0c52-e339-471c-9e4f-1521154f4881 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.191105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0c52-e339-471c-9e4f-1521154f4881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.193896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed0c52-e339-471c-9e4f-1521154f4881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.194741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0c52-e339-471c-9e4f-1521154f4881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:52.201063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed0c52-e339-471c-9e4f-1521154f4881 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:52.221631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 884/1752] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:52.224180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.225112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.225437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.275750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 869/1753] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 477 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:52.285275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:52.289134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:52.303197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.303570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.303865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.326511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 885/1754] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:52.327302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:52.376394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21335b8-8893-4f5e-b916-89bb4541f0f1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:52.393940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8585e4-361d-4469-a673-2e964b8fd407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:52.394871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8585e4-361d-4469-a673-2e964b8fd407 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.395161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8585e4-361d-4469-a673-2e964b8fd407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.403584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8585e4-361d-4469-a673-2e964b8fd407 admin admin] PATH_INFO: `/v3/projects/702938effa7e449e944f7e86b1a9c6cb/users/e90af3499f5d4374be024eb969dcd7c6/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.416056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:52.418576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:52.441686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 870/1755] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:52.446005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:52.469170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8585e4-361d-4469-a673-2e964b8fd407 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702938effa7e449e944f7e86b1a9c6cb, user_id=e90af3499f5d4374be024eb969dcd7c6, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:52.477670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d818956-c5de-4afd-930b-9c0bd4f8a566 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:52.478111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8585e4-361d-4469-a673-2e964b8fd407 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:52.495677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:52.506386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.506582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.518003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 886/1756] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:52 2022] PUT /identity/v3/projects/702938effa7e449e944f7e86b1a9c6cb/users/e90af3499f5d4374be024eb969dcd7c6/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:52.519424 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.520651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.520992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.521276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.522919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:52.523527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:52.529195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:52.567590 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 871/1757] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:52.569228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:52.594011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] MFA Rules not processed for user `e90af3499f5d4374be024eb969dcd7c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:52.595540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:52.599694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c711d9b-a7ee-4bb1-ada5-7fbcbbb75b35 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:52.709411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:52.744208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:52.791084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:52.792236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:52.792551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:52.792831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:52.802085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:52.802292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 887/1758] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:52.802562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:52.803449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:52.805624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:52.916836 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 872/1759] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:52 2022] POST /identity/v3/projects => generated 367 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:52.922208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.004789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e9542d-f89e-4ca8-8b6e-85ad483d713f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.139752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d414f1-ac40-4bff-96a0-b5f0ce0faf9a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.172506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.173475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.173937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.174361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.176127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.177059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.177499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.177925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.178781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.180882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.235664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.237082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.280743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 873/1760] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:53.282518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.298506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 888/1761] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:53.314093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.326326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fa3d4-31af-4d60-9e24-ddb4dec967ec service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.508321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1a75e-5b13-4758-a279-ccc2cbd008b5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.519962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.521367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.521632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.521830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.524729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.528933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.537260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.537260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.537260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.537260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.537615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.546696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.576531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 874/1762] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:53 2022] POST /identity/v3/users => generated 284 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:53.586541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.620931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 889/1763] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:53 2022] POST /identity/v3/users => generated 299 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:53.627740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.638167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3414d84-f59d-4a38-9642-45080d2eb6f5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.660978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.662557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.664025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.664331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.666497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.671890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f036f8-c424-4c27-9fdd-746bd071f171 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.672437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.684763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.686098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.686713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.687418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.688364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.690643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.693047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 875/1764] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:53.694428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.705852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4757-a8d5-4bf2-8bcd-41e44af3a202 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.708275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 890/1765] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:53.712365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.717559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.719126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.719751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.720904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.723469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.734184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9e314-8170-4518-a12f-bc76cc758158 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.739504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.756826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.759020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.760891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.761700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] PATH_INFO: `/v3/projects/08f2680699f74d2da88725ea52321cce/users/78d38a690b30494c80d4d3a9dbde4f5c/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.770846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 876/1766] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:53.772134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.785098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:53.794610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acf6d5c-55de-4ab2-8291-e3442f322c8f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:53.798333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f2680699f74d2da88725ea52321cce, user_id=78d38a690b30494c80d4d3a9dbde4f5c, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.814098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.816459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fc50b7-b659-4211-a598-8adb4f46a439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:53.817972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc50b7-b659-4211-a598-8adb4f46a439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.818482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc50b7-b659-4211-a598-8adb4f46a439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.818967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fc50b7-b659-4211-a598-8adb4f46a439 admin admin] PATH_INFO: `/v3/projects/4b93fdaf39d149f2b5b1e4f403335529/users/50b290dfacdf433fb2fd008aa7438be3/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:53.852894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc50b7-b659-4211-a598-8adb4f46a439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b93fdaf39d149f2b5b1e4f403335529, user_id=50b290dfacdf433fb2fd008aa7438be3, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:53.881521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fc50b7-b659-4211-a598-8adb4f46a439 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:53.898320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 891/1767] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:53 2022] PUT /identity/v3/projects/08f2680699f74d2da88725ea52321cce/users/78d38a690b30494c80d4d3a9dbde4f5c/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:01:53.902842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:53.914655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:53.946487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 877/1768] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:53 2022] PUT /identity/v3/projects/4b93fdaf39d149f2b5b1e4f403335529/users/50b290dfacdf433fb2fd008aa7438be3/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:53.949022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed019609-3a2d-41cb-bbb9-80beabae0fb5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:53.950078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed019609-3a2d-41cb-bbb9-80beabae0fb5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:53.950660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed019609-3a2d-41cb-bbb9-80beabae0fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.022324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ed019609-3a2d-41cb-bbb9-80beabae0fb5 None None] MFA Rules not processed for user `50b290dfacdf433fb2fd008aa7438be3`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:54.026754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ed019609-3a2d-41cb-bbb9-80beabae0fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:54.166264 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eff147-48aa-4847-afe1-688f4fb86a74 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:54.305796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 878/1769] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:54.308499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.310395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.310883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.341922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:54.343537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:54.420656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:54.422563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.423624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.424506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.428690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:54.439613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:54.453948 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 879/1770] 10.222.0.47 () {58 vars in 1015 bytes} [Mon Jun 20 06:01:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:54.455582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:54.470279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d5ddc-0a20-4f36-8bbe-cdc913b0abd0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:54.476752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:54.479132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.479697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.480086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.481845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:54.483997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:54.486408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 892/1771] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 584 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:54.487430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:54.491214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:54.534920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 880/1772] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:54 2022] POST /identity/v3/projects => generated 367 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:54.537769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:54.546589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77348fa8-d51b-466f-a7e3-1c3c8649cb9f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:54.688808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:54.689487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.689779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.690073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.693778 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d994a666-5894-4ee0-9e5d-a57b2e0f4b31 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:54.707103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:54.708621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:54.712511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8ddfe-3985-4983-899f-81346311f563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:54.715117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8ddfe-3985-4983-899f-81346311f563 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.718211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8ddfe-3985-4983-899f-81346311f563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.719234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8ddfe-3985-4983-899f-81346311f563 admin admin] PATH_INFO: `/v3/projects/08f2680699f74d2da88725ea52321cce/users/78d38a690b30494c80d4d3a9dbde4f5c/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.726942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 893/1773] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:54.727648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:54.728732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8ddfe-3985-4983-899f-81346311f563 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f2680699f74d2da88725ea52321cce, user_id=78d38a690b30494c80d4d3a9dbde4f5c, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:54.735543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8ddfe-3985-4983-899f-81346311f563 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:54.741785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0cb13-c495-4b85-b18b-46da6c958aef service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:54.747686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:54.749440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.749440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.749440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.749691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:54.761269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:54.772530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 881/1774] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:54 2022] PUT /identity/v3/projects/08f2680699f74d2da88725ea52321cce/users/78d38a690b30494c80d4d3a9dbde4f5c/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:54.784918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:54.785674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:54.786364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:54.802320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 894/1775] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:54 2022] POST /identity/v3/users => generated 290 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:54.807876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:54.846631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:54.879152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452c2e8-73e9-4c40-8ba2-1defdca600bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:54.881439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] MFA Rules not processed for user `78d38a690b30494c80d4d3a9dbde4f5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:54.883031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:54.920374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:55.034756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:55.035492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.036055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.038077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.038877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:55.040574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:55.084891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 895/1776] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:55.086059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:55.096374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.102491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 882/1777] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:55.103080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:55.129492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc3962-a8b4-457d-9d1e-729d56c27368 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:55.138389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e55076-1bfa-4f44-907b-229c97775dbf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:55.139842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:55.140888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.142195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.143990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] PATH_INFO: `/v3/projects/e93899145aaa458ea5b2eae78407e0cd/users/1e36fe821cf448b0bec655c6ad0ba89a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.163627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:55.164577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.164865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.165147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] PATH_INFO: `/v3/projects/c6c227432e8e4bf5b4003ce612b8822a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.167419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c227432e8e4bf5b4003ce612b8822a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:55.169874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:55.180420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93899145aaa458ea5b2eae78407e0cd, user_id=1e36fe821cf448b0bec655c6ad0ba89a, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:55.184397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:55.220986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 883/1778] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:55 2022] PUT /identity/v3/projects/e93899145aaa458ea5b2eae78407e0cd/users/1e36fe821cf448b0bec655c6ad0ba89a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:55.222375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:55.225836 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.331100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c227432e8e4bf5b4003ce612b8822a', 'request_id': 'req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:55.340391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c227432e8e4bf5b4003ce612b8822a', 'request_id': 'req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:55.353723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c227432e8e4bf5b4003ce612b8822a', 'request_id': 'req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:55.358764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c497cca2-3c70-4757-890d-e017e85affc0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:55.378073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] The token cache is being invalidate because project c6c227432e8e4bf5b4003ce612b8822a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:01:55.378897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:55.379943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:55.383740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 896/1779] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:55 2022] DELETE /identity/v3/projects/c6c227432e8e4bf5b4003ce612b8822a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:55.384777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:55.386447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.500475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46544f8d-1d0a-4edc-9565-36dcb3d82ab8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:55.510886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60717511-75d2-4674-a023-ebc81ad1575f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:55.511822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717511-75d2-4674-a023-ebc81ad1575f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.512113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717511-75d2-4674-a023-ebc81ad1575f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.512393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60717511-75d2-4674-a023-ebc81ad1575f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.515956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717511-75d2-4674-a023-ebc81ad1575f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:55.519908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60717511-75d2-4674-a023-ebc81ad1575f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:55.535325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:55.536403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.536927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.537443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.543279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.558422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 897/1780] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:55 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:55.560053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.560360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.560650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.585896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:55.587926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.605836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 898/1781] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:55.606831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:55.638086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f7c271-4ef5-4b5f-a1d9-8f33af2ebad0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:55.647282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:55.653550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db3500-9fbe-420e-8141-5adaa6be2d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:55.654479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db3500-9fbe-420e-8141-5adaa6be2d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.654765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db3500-9fbe-420e-8141-5adaa6be2d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.655040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db3500-9fbe-420e-8141-5adaa6be2d02 admin admin] PATH_INFO: `/v3/projects/e93899145aaa458ea5b2eae78407e0cd/users/1e36fe821cf448b0bec655c6ad0ba89a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.669740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db3500-9fbe-420e-8141-5adaa6be2d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93899145aaa458ea5b2eae78407e0cd, user_id=1e36fe821cf448b0bec655c6ad0ba89a, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:55.680023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db3500-9fbe-420e-8141-5adaa6be2d02 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:55.704761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:55.706552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eb1be3-5141-476c-a64d-b9cfa3db6c81 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:55.710200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 899/1782] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:55 2022] PUT /identity/v3/projects/e93899145aaa458ea5b2eae78407e0cd/users/1e36fe821cf448b0bec655c6ad0ba89a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:55.717911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.720564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.720981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.747946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 884/1783] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:55 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:55.751059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:55.752730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:55.753234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:55.786592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:55.791194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.816470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:55.836617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] MFA Rules not processed for user `1e36fe821cf448b0bec655c6ad0ba89a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:55.840177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:55.999505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 900/1784] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:56.000105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:56.003703 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:56.028185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7ed159-fbdf-4e84-967e-63a640e0d2e5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:56.040693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:56.041766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:56.042162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:56.048538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] PATH_INFO: `/v3/users/e90af3499f5d4374be024eb969dcd7c6` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:56.048931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 885/1785] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:56.048931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:56.049486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e90af3499f5d4374be024eb969dcd7c6)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:56.053770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:56.076017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05988989-9a17-4a6a-84ff-33c75b09664b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:56.086168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:56.087395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:56.088642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:56.089798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:56.090805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:01:56.093410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:56.098390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:56.129030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 886/1786] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:01:56.133380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:56.139968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:56.156760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e90af3499f5d4374be024eb969dcd7c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:56.186815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e90af3499f5d4374be024eb969dcd7c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:56.187776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e90af3499f5d4374be024eb969dcd7c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:56.189198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e90af3499f5d4374be024eb969dcd7c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:56.214705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e90af3499f5d4374be024eb969dcd7c6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:01:56.266715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 901/1787] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:01:55 2022] DELETE /identity/v3/users/e90af3499f5d4374be024eb969dcd7c6 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:56.267774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:56.271120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:56.354669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844acb01-b0d1-4474-9a0c-d0ed92027fb4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:56.481473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8714b-ce2a-4bdf-92f9-11dd23ff2553 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:56.493942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759c68e7-c384-4433-bed7-e1dc158e85bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:56.500665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c68e7-c384-4433-bed7-e1dc158e85bc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:56.501281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c68e7-c384-4433-bed7-e1dc158e85bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:56.503516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759c68e7-c384-4433-bed7-e1dc158e85bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:56.504390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c68e7-c384-4433-bed7-e1dc158e85bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:56.511630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759c68e7-c384-4433-bed7-e1dc158e85bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:56.528269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:56.529065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:56.529360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:56.529700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:56.540557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:56.589154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 902/1788] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:56 2022] POST /identity/v3/projects => generated 379 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:56.592834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:56.594637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:56.595051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:56.620469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:56.641806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:56.643417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:56.664117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:56.736296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:56.737696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:56.759939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 887/1789] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:56.761480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:56.788795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2160e235-c430-49a0-a1a5-01ed0ee13d62 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:56.886120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 903/1790] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:56.955405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:56.956849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:56.989454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:56.990168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:56.990457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:56.990811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:56.991476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:56.993876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:57.055039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 888/1791] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:56 2022] POST /identity/v3/users => generated 290 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:57.058585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:57.062622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:57.114725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0307cb9-29cd-404a-99af-a791c410a3ea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:57.125174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:57.125858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.126974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.127438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.136436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:57.231019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc293a16-c63c-424c-b62e-3639b9462c82 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:57.249507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:57.251097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.252215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.255353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.297090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:57.298567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:57.315490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:57.319842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:57.344846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 904/1792] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:56 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:01:57.345315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 889/1793] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:57.346196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:57.372345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bae6335-e27d-4cc8-b829-ccf0baa25f9f service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:57.389527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1d6b0-8e95-4b22-b6ce-40c53f0ef228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:57.389527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d6b0-8e95-4b22-b6ce-40c53f0ef228 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.389527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d6b0-8e95-4b22-b6ce-40c53f0ef228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.389527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1d6b0-8e95-4b22-b6ce-40c53f0ef228 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.389958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1d6b0-8e95-4b22-b6ce-40c53f0ef228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:57.390582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1d6b0-8e95-4b22-b6ce-40c53f0ef228 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:57.415296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 905/1794] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:57.420453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:57.435161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb3fa0-eac0-4cd8-99c4-9be38970e034 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:57.467671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:57.468383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.468716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.468951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.469702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:57.471384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:57.475409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7018-0d4a-4bef-89b9-edeb66bf52cf None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.475926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7018-0d4a-4bef-89b9-edeb66bf52cf None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.476352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f7018-0d4a-4bef-89b9-edeb66bf52cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.499924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 890/1795] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:57.501078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:57.540540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9b5f7018-0d4a-4bef-89b9-edeb66bf52cf None None] MFA Rules not processed for user `70323e3276ca4aaa87f55c5fb38837aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:57.543909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9b5f7018-0d4a-4bef-89b9-edeb66bf52cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:57.558811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb1c86-25eb-4289-b408-29ef325eb8bc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:57.573440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:57.574533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.574824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.575108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] PATH_INFO: `/v3/projects/bc10c13691bf4c64ac78c7fe73da5b5b/users/f7e8a9cda3af43a19d7c355b050e706d/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.640327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc10c13691bf4c64ac78c7fe73da5b5b, user_id=f7e8a9cda3af43a19d7c355b050e706d, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:57.645572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:57.675462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 906/1796] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:57.716524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 891/1797] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:57 2022] PUT /identity/v3/projects/bc10c13691bf4c64ac78c7fe73da5b5b/users/f7e8a9cda3af43a19d7c355b050e706d/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:57.730446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:57.730849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:57.731135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:57.742160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:57.786308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] MFA Rules not processed for user `f7e8a9cda3af43a19d7c355b050e706d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:57.787886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:57.978870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 907/1798] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:57.982417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:58.026269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:58.234737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a60e26-3c67-4815-9fdf-d4fd4a7095bc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:58.240477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:58.241204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.241516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.241827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.242556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:58.246618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:58.314561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 892/1799] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:01:57 2022] POST /identity/v3/projects => generated 379 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:58.315800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:58.331930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c88a87-be7b-4937-b0bc-fd9c6d3f53df None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:58.432936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:58.433764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.434390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.434648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.435782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:58.438590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:58.483455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:58.501765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 908/1800] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:01:58 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:01:58.508982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850407f2-941d-4afa-8196-f14075c2dd3e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:58.524313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:58.526513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.527070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.528540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.529528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:58.531927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:58.582914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:58.594829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 893/1801] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:58.604464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d56cba-1596-4fc2-bd2c-2c69e0a0a2ad admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:58.611661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:58.612912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.613726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.614690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] PATH_INFO: `/v3/projects/5af4629a10be48f3a1866630c44a602c/users/4a7b41c0bdfd4940a08ee485b6c6b101/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.642771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af4629a10be48f3a1866630c44a602c, user_id=4a7b41c0bdfd4940a08ee485b6c6b101, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:58.648789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:58.674943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 909/1802] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:58 2022] PUT /identity/v3/projects/5af4629a10be48f3a1866630c44a602c/users/4a7b41c0bdfd4940a08ee485b6c6b101/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:58.676420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:58.693230 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc03a91b-882e-4298-a405-5a36f8dee39d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:58.838008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89787ea-7f3f-4ac2-91a5-bd57269227fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:58.839090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89787ea-7f3f-4ac2-91a5-bd57269227fb admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.839619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89787ea-7f3f-4ac2-91a5-bd57269227fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.840119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89787ea-7f3f-4ac2-91a5-bd57269227fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.841068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89787ea-7f3f-4ac2-91a5-bd57269227fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:58.843378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89787ea-7f3f-4ac2-91a5-bd57269227fb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:58.868821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 894/1803] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:01:58.873788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:58.885982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dcebe4-795a-4fbb-b33b-df7cb207f1ea admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:58.929577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:58.930666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.931036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.931370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] PATH_INFO: `/v3/projects/5af4629a10be48f3a1866630c44a602c/users/4a7b41c0bdfd4940a08ee485b6c6b101/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.956322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] RBAC: Authorizing `identity:create_grant(project_id=5af4629a10be48f3a1866630c44a602c, user_id=4a7b41c0bdfd4940a08ee485b6c6b101, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:58.959039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:58.964009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16c44-9b79-455c-9695-cf5880d62ea3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:58.964305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16c44-9b79-455c-9695-cf5880d62ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:58.964588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae16c44-9b79-455c-9695-cf5880d62ea3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:58.969936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 895/1804] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:01:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:01:59.011286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 910/1805] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:01:58 2022] PUT /identity/v3/projects/5af4629a10be48f3a1866630c44a602c/users/4a7b41c0bdfd4940a08ee485b6c6b101/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:01:59.026193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6c4ac-5679-42d8-9bad-83c7b2758f20 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:59.026551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6c4ac-5679-42d8-9bad-83c7b2758f20 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:59.027016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec6c4ac-5679-42d8-9bad-83c7b2758f20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:59.127749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fec6c4ac-5679-42d8-9bad-83c7b2758f20 None None] MFA Rules not processed for user `4a7b41c0bdfd4940a08ee485b6c6b101`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:59.129103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fec6c4ac-5679-42d8-9bad-83c7b2758f20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:59.390016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 896/1806] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:01:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:01:59.402384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:59.404223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:59.538192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:59.569376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfda812-c5b0-4a7f-af80-bb02730b73db admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:01:59.581351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:01:59.582363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:59.582869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:59.583435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:59.611547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:01:59.612408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:01:59.613019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:01:59.614370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:01:59.621036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:01:59.632732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:01:59.648734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:01:59.650113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:59.663945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 911/1807] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:01:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:01:59.689913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:01:59.691793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:01:59.892722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 897/1808] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:01:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:01:59.975032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab9b6f8-5ba1-46bf-8942-3d9d18540f3e service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:00.014641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:00.015854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:00.017767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:00.018294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] PATH_INFO: `/v3/projects/bcb94fc880224954ba8aa9fb88d647c2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:00.022267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb94fc880224954ba8aa9fb88d647c2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:00.027525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:00.159761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb94fc880224954ba8aa9fb88d647c2', 'request_id': 'req-763c4add-4c1b-447c-a4f6-0d28a65784a1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.168592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb94fc880224954ba8aa9fb88d647c2', 'request_id': 'req-763c4add-4c1b-447c-a4f6-0d28a65784a1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.177793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb94fc880224954ba8aa9fb88d647c2', 'request_id': 'req-763c4add-4c1b-447c-a4f6-0d28a65784a1'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.210018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] The token cache is being invalidate because project bcb94fc880224954ba8aa9fb88d647c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:00.210502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.211178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.216436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 912/1809] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:01:59 2022] DELETE /identity/v3/projects/bcb94fc880224954ba8aa9fb88d647c2 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:00.218143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:00.243429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f8861f-bb27-4e34-9b79-20c77065459d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:00.402681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:00.403548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:00.404333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:00.411728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] PATH_INFO: `/v3/users/b442ef2b6806452286e8400fbf5c67ce` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:00.413375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b442ef2b6806452286e8400fbf5c67ce)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:00.415676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:00.515583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b442ef2b6806452286e8400fbf5c67ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.533715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b442ef2b6806452286e8400fbf5c67ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.534539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b442ef2b6806452286e8400fbf5c67ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.535245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b442ef2b6806452286e8400fbf5c67ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.574934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b442ef2b6806452286e8400fbf5c67ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:00.597761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 898/1810] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:00 2022] DELETE /identity/v3/users/b442ef2b6806452286e8400fbf5c67ce => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:00.668579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:00.671216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:00.797604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:00.799214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:00.851618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:00.882655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c4add-4c1b-447c-a4f6-0d28a65784a1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:00.951950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:01.028860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a7b41c0bdfd4940a08ee485b6c6b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5af4629a10be48f3a1866630c44a602c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:01.029824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:01.030406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:01.030773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:01.032800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:01.033330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:01.035313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:01.053381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 913/1811] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:01.058915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e75e03-8373-4198-b8b0-4e85c0f33e32 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:01.061756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:01.068504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:01.069930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:01.071211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:01.071768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:01.072304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:01.075814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:01.102503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:01.105452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:01.108559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c0e6-78cb-4c9e-8abf-217ba1a7b0a2 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:01.114581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a7b41c0bdfd4940a08ee485b6c6b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5af4629a10be48f3a1866630c44a602c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:01.115462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:01.115900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:01.116425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:01.117314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:01.120395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:01.147952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 899/1812] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:01.159415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 914/1813] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:01 2022] POST /identity/v3/projects => generated 354 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:01.163359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:01.178251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3929765f-3848-4327-87b8-d1bd326865c2 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:01.267607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a7b41c0bdfd4940a08ee485b6c6b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5af4629a10be48f3a1866630c44a602c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:01.268346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:01.268649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:01.268871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:01.270347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:01.270820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:01.272976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:01.290834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 900/1814] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:01.395590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:01.412070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57237f26-e93a-4fa0-9d77-ba9ac2415431 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:01.423542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a7b41c0bdfd4940a08ee485b6c6b101', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5af4629a10be48f3a1866630c44a602c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:01.424500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:01.425945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:01.426996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] PATH_INFO: `/v3/projects/a881196e41b94e01946c8b27462c6e0a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:01.429990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorizing `identity:delete_project(project_id=a881196e41b94e01946c8b27462c6e0a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:01.434594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:01.571699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a881196e41b94e01946c8b27462c6e0a', 'request_id': 'req-eeeedb72-a6ca-45e2-9542-70734102b18d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:01.583023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a881196e41b94e01946c8b27462c6e0a', 'request_id': 'req-eeeedb72-a6ca-45e2-9542-70734102b18d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:01.640721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a881196e41b94e01946c8b27462c6e0a', 'request_id': 'req-eeeedb72-a6ca-45e2-9542-70734102b18d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:01.701460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] The token cache is being invalidate because project a881196e41b94e01946c8b27462c6e0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:01.702547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:01.703708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:01.706492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 915/1815] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:01 2022] DELETE /identity/v3/projects/a881196e41b94e01946c8b27462c6e0a => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:02.285256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:02.287981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:02.297448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:02.299421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:02.548272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2a250-f6d6-4942-bcaf-b42e5f6c6c58 tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:02.565189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:02.566372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:02.566847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:02.567280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] PATH_INFO: `/v3/users/f7e8a9cda3af43a19d7c355b050e706d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:02.570720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7e8a9cda3af43a19d7c355b050e706d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:02.574787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:02.575447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedb72-a6ca-45e2-9542-70734102b18d tempest-VolumeQuotaClassesTest-725089055 tempest-VolumeQuotaClassesTest-725089055-project-admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:02.596635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:02.598113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:02.599082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:02.599587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] PATH_INFO: `/v3/projects/702938effa7e449e944f7e86b1a9c6cb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:02.601522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] RBAC: Authorizing `identity:delete_project(project_id=702938effa7e449e944f7e86b1a9c6cb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:02.605121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:02.699612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:02.703030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:02.738641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7e8a9cda3af43a19d7c355b050e706d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.775074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702938effa7e449e944f7e86b1a9c6cb', 'request_id': 'req-c2681ab1-b6b7-4424-9694-343bf5c6425b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.782646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7e8a9cda3af43a19d7c355b050e706d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.787646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7e8a9cda3af43a19d7c355b050e706d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.790268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7e8a9cda3af43a19d7c355b050e706d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.803931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702938effa7e449e944f7e86b1a9c6cb', 'request_id': 'req-c2681ab1-b6b7-4424-9694-343bf5c6425b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.827120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702938effa7e449e944f7e86b1a9c6cb', 'request_id': 'req-c2681ab1-b6b7-4424-9694-343bf5c6425b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.842867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7e8a9cda3af43a19d7c355b050e706d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.867486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] The token cache is being invalidate because project 702938effa7e449e944f7e86b1a9c6cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:02.868072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.870495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:02.884959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 916/1816] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:02 2022] DELETE /identity/v3/projects/702938effa7e449e944f7e86b1a9c6cb => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:02.885980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:02.889500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:02.895907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 901/1817] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:02 2022] DELETE /identity/v3/users/f7e8a9cda3af43a19d7c355b050e706d => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:02.896654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:02.899230 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:03.062074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2681ab1-b6b7-4424-9694-343bf5c6425b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:03.201023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cf2b3-653f-4ad9-a3f5-0d0edba8ee7e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:03.235622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:03.252048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:03.253689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:03.254085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] PATH_INFO: `/v3/users/78d38a690b30494c80d4d3a9dbde4f5c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:03.256662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] RBAC: Authorizing `identity:delete_user(user_id=78d38a690b30494c80d4d3a9dbde4f5c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:03.259748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:03.289047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:03.295447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:03.295447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:03.295447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] PATH_INFO: `/v3/projects/79ef203fdee84e0ab1466f8a90d8820b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:03.297929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79ef203fdee84e0ab1466f8a90d8820b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:03.303941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:03.393664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78d38a690b30494c80d4d3a9dbde4f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.415680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78d38a690b30494c80d4d3a9dbde4f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.420362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78d38a690b30494c80d4d3a9dbde4f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.423636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78d38a690b30494c80d4d3a9dbde4f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.447071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79ef203fdee84e0ab1466f8a90d8820b', 'request_id': 'req-ba65ad6f-52f0-4177-8315-aa9dbc74e919'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.457911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-43d77e38-b343-4956-ba73-35c5d594065e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78d38a690b30494c80d4d3a9dbde4f5c'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.463863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79ef203fdee84e0ab1466f8a90d8820b', 'request_id': 'req-ba65ad6f-52f0-4177-8315-aa9dbc74e919'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.465639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:03.485256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79ef203fdee84e0ab1466f8a90d8820b', 'request_id': 'req-ba65ad6f-52f0-4177-8315-aa9dbc74e919'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.516778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] The token cache is being invalidate because project 79ef203fdee84e0ab1466f8a90d8820b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:03.518642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.519990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ba65ad6f-52f0-4177-8315-aa9dbc74e919 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:03.526397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 917/1818] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:02 2022] DELETE /identity/v3/projects/79ef203fdee84e0ab1466f8a90d8820b => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:03.527065 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 902/1819] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:02 2022] DELETE /identity/v3/users/78d38a690b30494c80d4d3a9dbde4f5c => generated 0 bytes in 633 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:03.533643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77296a-099e-4199-b864-cb670a0691dd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:03.534005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:03.536109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:03.536109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:03.536486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77296a-099e-4199-b864-cb670a0691dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:03.536740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a77296a-099e-4199-b864-cb670a0691dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:03.597211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7a77296a-099e-4199-b864-cb670a0691dd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:03.598484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7a77296a-099e-4199-b864-cb670a0691dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:03.600468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:03.605008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:03.730709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:03.766187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 918/1820] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:03.810370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 903/1821] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:03.830179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:03.830669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:03.830756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:03.866750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:03.868673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:03.881127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 919/1822] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:03.886047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:03.923692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deccd6a2-2bfe-4657-b587-a1ffd1595cb2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:03.929002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:03.929658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:03.929990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:03.930276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:03.931137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:03.931433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:03.933366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:03.971078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 904/1823] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:03.975282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:03.982537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:03.987398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:03.994415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c20c594-56d9-4f1f-9f00-4b3ab8c4d41b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.025548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.026297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.026589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.030042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.030837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.032909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.097549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 920/1824] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:03 2022] POST /identity/v3/projects => generated 357 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:04.098525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:04.147659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d27984-9722-4d1d-9d29-6611713203b9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.358598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271caade-cc07-4c98-abfe-1a22bbf001f7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.364417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dffb86-0957-4f56-bbb9-22b201419258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.365237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffb86-0957-4f56-bbb9-22b201419258 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.365668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffb86-0957-4f56-bbb9-22b201419258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.366079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dffb86-0957-4f56-bbb9-22b201419258 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.366794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffb86-0957-4f56-bbb9-22b201419258 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.369786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dffb86-0957-4f56-bbb9-22b201419258 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.396352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.407911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.408286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.408560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.411460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 921/1825] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:04 2022] POST /identity/v3/users => generated 279 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:04.413195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0568a-f976-4582-92ea-9ad2e521eb28 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.414939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0568a-f976-4582-92ea-9ad2e521eb28 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.415498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0568a-f976-4582-92ea-9ad2e521eb28 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.419191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 922/1826] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:02:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:02:04.421489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0568a-f976-4582-92ea-9ad2e521eb28 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:04.450044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0568a-f976-4582-92ea-9ad2e521eb28 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.463431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3a4f1-287d-4db9-bf3a-848fdfb18996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.464604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3a4f1-287d-4db9-bf3a-848fdfb18996 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.465160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3a4f1-287d-4db9-bf3a-848fdfb18996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.467234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3a4f1-287d-4db9-bf3a-848fdfb18996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.468186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3a4f1-287d-4db9-bf3a-848fdfb18996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.470015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3a4f1-287d-4db9-bf3a-848fdfb18996 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.475462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.476920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.506720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 923/1827] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:04.520556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3c01-09f5-472b-8f53-10f56a058613 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.520923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3c01-09f5-472b-8f53-10f56a058613 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.521219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a3c01-09f5-472b-8f53-10f56a058613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.521343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 905/1828] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:04.522204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:04.547560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0fa6f-c46b-47d9-828e-c2c94dbc6093 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.563186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.563405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd7a3c01-09f5-472b-8f53-10f56a058613 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:04.563916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.564205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.564481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.564597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd7a3c01-09f5-472b-8f53-10f56a058613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:04.565209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.571261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.601441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 906/1829] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:04.608023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:04.627910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74e6b20-89a6-46cd-b052-9644ac284715 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.651217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.652205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.652494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.652774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] PATH_INFO: `/v3/projects/5b9e254898004b09b4cb885636734787/users/40d24282ecb94732b39cafe72820590b/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.669712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:04.704658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b9e254898004b09b4cb885636734787, user_id=40d24282ecb94732b39cafe72820590b, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.707501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.727918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 924/1830] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:02:04 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:04.730056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37013f5b-cb54-40d0-b308-073e934c2d01 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.730475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37013f5b-cb54-40d0-b308-073e934c2d01 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.730769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37013f5b-cb54-40d0-b308-073e934c2d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.753767 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 907/1831] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:04 2022] PUT /identity/v3/projects/5b9e254898004b09b4cb885636734787/users/40d24282ecb94732b39cafe72820590b/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:04.756049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:04.768181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:04.791852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-37013f5b-cb54-40d0-b308-073e934c2d01 None None] MFA Rules not processed for user `adab7140fc804689af945257511cfded`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:04.793867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-37013f5b-cb54-40d0-b308-073e934c2d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:04.827188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:04.855459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913c7974-58e7-485b-9ab6-8fdfa1d24241 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:04.862201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:04.862929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.863279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.863571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.875069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:04.877219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:04.890490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 908/1832] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:04.894138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:04.895803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:04.906978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 925/1833] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:04.908766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:04.909140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:04.909630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:04.969472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] MFA Rules not processed for user `40d24282ecb94732b39cafe72820590b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:04.971180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.032681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6fddf-73e6-4dd1-aad3-ecb44ce6a900 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.039093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.040303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.040680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.041376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] PATH_INFO: `/v3/users/50b290dfacdf433fb2fd008aa7438be3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.043558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b290dfacdf433fb2fd008aa7438be3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.046822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.128287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b290dfacdf433fb2fd008aa7438be3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.136826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b290dfacdf433fb2fd008aa7438be3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.137585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b290dfacdf433fb2fd008aa7438be3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.138297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b290dfacdf433fb2fd008aa7438be3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.159684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-36f612fe-6104-4d97-9efb-2e243a5bbcd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b290dfacdf433fb2fd008aa7438be3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.186814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 926/1834] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:04 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:05.191485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.193445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.216480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504d6d5e-b847-4849-ada1-d2c76ed654ec None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.222164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 909/1835] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:04 2022] DELETE /identity/v3/users/50b290dfacdf433fb2fd008aa7438be3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:05.225614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.225614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.225614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.256848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:05.258864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.312122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.313309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.318085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.318661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.319707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.322040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.330690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:05.346482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:05.357072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 910/1836] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:05.373827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 927/1837] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:05 2022] POST /identity/v3/projects => generated 357 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:05.378210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.381881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.386181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.394270 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de551d0-f39e-4e66-ad80-9b6511760e48 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.500119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.502959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.503455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.503899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.504754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.509801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.515861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900bc03-5491-47a7-b159-b8697967c11e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.522023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.522023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.522023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.522387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.533784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.535286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.551888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 928/1838] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:05.553766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.556452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.568374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 911/1839] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:05 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:05.571761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.578915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eefc98-631e-408f-9580-170f62ae4a34 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.583720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9632bcf2-4c19-4de6-be1d-cfd6d25325a8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.586269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.589736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.590504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.591801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.591801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.592190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.592872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.593116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.593480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] PATH_INFO: `/v3/projects/08f2680699f74d2da88725ea52321cce` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.594985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.609741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 912/1840] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:05.613925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.616155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f2680699f74d2da88725ea52321cce)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.618089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.627208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b369d608-dbbd-491f-8e73-a7ebbf8527dc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.642425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.643644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.644286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.644511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] PATH_INFO: `/v3/projects/ee5cb432ab4644af99a5123a6586ca55/users/81a196eedb864611b291c3d3bd47b5f1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.677515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5cb432ab4644af99a5123a6586ca55, user_id=81a196eedb864611b291c3d3bd47b5f1, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.683114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.691997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f2680699f74d2da88725ea52321cce', 'request_id': 'req-5e0689de-690b-47a7-a72f-1a475cfc0e58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.699726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f2680699f74d2da88725ea52321cce', 'request_id': 'req-5e0689de-690b-47a7-a72f-1a475cfc0e58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.713452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f2680699f74d2da88725ea52321cce', 'request_id': 'req-5e0689de-690b-47a7-a72f-1a475cfc0e58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.717527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 913/1841] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:05 2022] PUT /identity/v3/projects/ee5cb432ab4644af99a5123a6586ca55/users/81a196eedb864611b291c3d3bd47b5f1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:05.721343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.742148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] The token cache is being invalidate because project 08f2680699f74d2da88725ea52321cce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:05.742561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.743667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e0689de-690b-47a7-a72f-1a475cfc0e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:05.753035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 929/1842] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:05 2022] DELETE /identity/v3/projects/08f2680699f74d2da88725ea52321cce => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:05.756101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d65ec3-3831-4c6e-a991-1ce3200ed626 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.756795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d65ec3-3831-4c6e-a991-1ce3200ed626 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.757502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d65ec3-3831-4c6e-a991-1ce3200ed626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.781719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d6d65ec3-3831-4c6e-a991-1ce3200ed626 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:05.783776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6d65ec3-3831-4c6e-a991-1ce3200ed626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.850086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda340f-da49-4ec3-adc8-09fcfa6e9c0d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.856675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.857518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.857765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.858035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.858936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.860708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.877774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 914/1843] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:05.880958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.883359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.885818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 930/1844] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:05.894619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3869768-8a65-4729-a593-474204da9f8e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.894927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3869768-8a65-4729-a593-474204da9f8e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.895209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3869768-8a65-4729-a593-474204da9f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.906409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ee3569-287e-4c98-a5e0-c54edb82a10e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:05.911923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:05.913912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:05.914303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:05.914787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] PATH_INFO: `/v3/projects/ee5cb432ab4644af99a5123a6586ca55/users/81a196eedb864611b291c3d3bd47b5f1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:05.920666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5cb432ab4644af99a5123a6586ca55, user_id=81a196eedb864611b291c3d3bd47b5f1, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:05.924053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:05.924867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b3869768-8a65-4729-a593-474204da9f8e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:05.926407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3869768-8a65-4729-a593-474204da9f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:05.941506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 915/1845] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:05 2022] PUT /identity/v3/projects/ee5cb432ab4644af99a5123a6586ca55/users/81a196eedb864611b291c3d3bd47b5f1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:05.942629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:05.953611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:05.975510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.056244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 931/1846] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:06.057801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.058096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.059108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.085665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:06.121604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] MFA Rules not processed for user `81a196eedb864611b291c3d3bd47b5f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:06.126669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.184209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd15326-2f2b-4af0-9423-9c982bfff67d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:06.199732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:06.201208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.204601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.205299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] PATH_INFO: `/v3/projects/bc10c13691bf4c64ac78c7fe73da5b5b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.208387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc10c13691bf4c64ac78c7fe73da5b5b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:06.213467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:06.296835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 932/1847] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:06 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:06.297741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:06.327308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc10c13691bf4c64ac78c7fe73da5b5b', 'request_id': 'req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.330489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e95d3-5d8b-4dd4-bbd7-af8c0200633d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:06.340046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc10c13691bf4c64ac78c7fe73da5b5b', 'request_id': 'req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.352925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc10c13691bf4c64ac78c7fe73da5b5b', 'request_id': 'req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.377291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] The token cache is being invalidate because project bc10c13691bf4c64ac78c7fe73da5b5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:06.377845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.378890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.380991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 916/1848] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:05 2022] DELETE /identity/v3/projects/bc10c13691bf4c64ac78c7fe73da5b5b => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:06.383718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:06.386774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.476007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:06.476442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.476734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.477008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.477744 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:06.478048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:06.479829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:06.517463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 933/1849] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:06.519459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:06.526678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.533903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0ba5a-0a96-4cab-b80c-cc2c28ad5c98 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:06.544327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:06.545143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.545568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.545855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.549757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c232a0-2b56-4fef-a2e2-6b8157fddca8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:06.551789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.562565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:06.563515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.563799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.564073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] PATH_INFO: `/v3/users/4a7b41c0bdfd4940a08ee485b6c6b101` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.566462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a7b41c0bdfd4940a08ee485b6c6b101)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:06.568760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:06.637247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:06.659301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a7b41c0bdfd4940a08ee485b6c6b101'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.673723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a7b41c0bdfd4940a08ee485b6c6b101'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.674689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a7b41c0bdfd4940a08ee485b6c6b101'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.675436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a7b41c0bdfd4940a08ee485b6c6b101'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.694871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a7b41c0bdfd4940a08ee485b6c6b101'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:06.718015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 934/1850] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:06 2022] DELETE /identity/v3/users/4a7b41c0bdfd4940a08ee485b6c6b101 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:06.719101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:06.720841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.731623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:06.733414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:06.746844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 917/1851] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:06 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:06.748070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:06.749725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:06.765203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a456da96-96d5-4953-803a-9c7b813950c4 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:06.771125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:06.882474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:06.885369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.886523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.888238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] PATH_INFO: `/v3/projects/4b93fdaf39d149f2b5b1e4f403335529` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.890795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b93fdaf39d149f2b5b1e4f403335529)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:06.896442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:06.897100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022afd5-96a7-41a3-8d04-fd9cbf5f83bf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:06.910590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:06.912035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:06.912496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:06.913712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:06.917084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:06.920189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:06.994936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 935/1852] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:06 2022] POST /identity/v3/projects => generated 383 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:06.996096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:06.999511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b93fdaf39d149f2b5b1e4f403335529', 'request_id': 'req-21aa0f6e-7026-4148-b180-495e80f0efc6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.001057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.006700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b93fdaf39d149f2b5b1e4f403335529', 'request_id': 'req-21aa0f6e-7026-4148-b180-495e80f0efc6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.016026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b93fdaf39d149f2b5b1e4f403335529', 'request_id': 'req-21aa0f6e-7026-4148-b180-495e80f0efc6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.064366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] The token cache is being invalidate because project 4b93fdaf39d149f2b5b1e4f403335529 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:07.065320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.068950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.073779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 918/1853] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:06 2022] DELETE /identity/v3/projects/4b93fdaf39d149f2b5b1e4f403335529 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:07.077384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.084809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.120761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eed2b04-cd34-4404-8f6c-dd162504dd04 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.124854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.125947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.126422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.126772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.129020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.242631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:07.256659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aa0f6e-7026-4148-b180-495e80f0efc6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.269673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.272312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.298410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 936/1854] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:06 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:07.299600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.301736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.348661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:07.386544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.387145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.387448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.387736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.388544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.390388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.407699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574e927f-1cea-4285-b5c0-1e4f18341f56 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.415310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.420587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.420936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.421228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] PATH_INFO: `/v3/users/1e36fe821cf448b0bec655c6ad0ba89a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.423293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e36fe821cf448b0bec655c6ad0ba89a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.425648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.430920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 919/1855] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:07 2022] POST /identity/v3/users => generated 292 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:07.435498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.448829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c728d-f97d-45b7-aea1-fe9cdfcf1fe3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.455732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.458505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.459146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.459655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.460577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.462431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.476045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 920/1856] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:07.480371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.492710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da494161-6ed5-42cc-8f0e-820ae5f3c0ea admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.495917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e36fe821cf448b0bec655c6ad0ba89a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.499110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.500915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.501553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.502028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.502971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.503838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e36fe821cf448b0bec655c6ad0ba89a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.504552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e36fe821cf448b0bec655c6ad0ba89a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.505684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e36fe821cf448b0bec655c6ad0ba89a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.505832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.526224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 921/1857] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:07.529183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.531779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.538583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-90246d1b-8cfe-4c2a-acfa-09af26679230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e36fe821cf448b0bec655c6ad0ba89a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.549404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa0cd63-ff0b-4669-b6d4-d41ac897a8bf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.556475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 937/1858] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:07 2022] DELETE /identity/v3/users/1e36fe821cf448b0bec655c6ad0ba89a => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:07.563809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.566499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.566930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.567220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] PATH_INFO: `/v3/projects/036bb651fb1b4817bc8f6593293a9d2c/users/1e4e470e9fe34059913a6acf84a05435/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.608980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=036bb651fb1b4817bc8f6593293a9d2c, user_id=1e4e470e9fe34059913a6acf84a05435, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.612600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.640872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 922/1859] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:07 2022] PUT /identity/v3/projects/036bb651fb1b4817bc8f6593293a9d2c/users/1e4e470e9fe34059913a6acf84a05435/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:07.653196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.653633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.654021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.688212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] MFA Rules not processed for user `1e4e470e9fe34059913a6acf84a05435`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:07.689435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.760070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:07.774778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 938/1860] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:07.779563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.875355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ecafe3-8064-4e95-8af3-de5bebbec9b4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.876640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.879206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:07.882262 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.883417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.883417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.883932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.884833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.886899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.906706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8a3d04-0884-4513-95f4-6e4913f48e21 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:07.911918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:07.912887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:07.913293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:07.913638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] PATH_INFO: `/v3/projects/5af4629a10be48f3a1866630c44a602c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:07.915100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5af4629a10be48f3a1866630c44a602c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:07.917236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:07.925465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 923/1861] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:07 2022] POST /identity/v3/projects => generated 383 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:07.931056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:07.979763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5af4629a10be48f3a1866630c44a602c', 'request_id': 'req-ab808bff-1d84-4f41-a751-118ab276194b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.987991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5af4629a10be48f3a1866630c44a602c', 'request_id': 'req-ab808bff-1d84-4f41-a751-118ab276194b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:07.994950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5af4629a10be48f3a1866630c44a602c', 'request_id': 'req-ab808bff-1d84-4f41-a751-118ab276194b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.005020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1d35d8-2728-4f7f-8d8c-aa45d9a34864 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.011370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] The token cache is being invalidate because project 5af4629a10be48f3a1866630c44a602c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:08.012068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.012849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ab808bff-1d84-4f41-a751-118ab276194b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.014765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 939/1862] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:07 2022] DELETE /identity/v3/projects/5af4629a10be48f3a1866630c44a602c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:08.019505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65769646-229d-4445-a2d9-af269d0a7f5c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.019799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65769646-229d-4445-a2d9-af269d0a7f5c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.020131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65769646-229d-4445-a2d9-af269d0a7f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.027407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:08.040365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-65769646-229d-4445-a2d9-af269d0a7f5c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:08.041800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-65769646-229d-4445-a2d9-af269d0a7f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.077733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.078716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.078716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.078927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.081021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.083373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.114312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 924/1863] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:07 2022] POST /identity/v3/users => generated 298 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:08.123162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 940/1864] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:08.123349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.125234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.125325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.125433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.125718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.142241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc380580-e735-490d-a21d-a2bbe8d54146 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.147712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:08.147918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.148366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.148785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.149284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.149728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.151120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.152827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.165083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 941/1865] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:08.168729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 925/1866] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:08.172447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.178230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.185922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b481a-5d66-434b-965d-342d7f834d8d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.187201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e981664d-7320-413a-a20c-8e0a075eb895 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.193442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.193610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.193730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.193730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.193730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] PATH_INFO: `/v3/projects/5f6fe842ddba4f79a507c15eba850ceb/users/795dbdfdc28a4d77a4860e764496bcd2/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.193962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.193962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.193962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.194546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:08.194862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.197704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.215312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 942/1867] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:02:08.217010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.221389 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6fe842ddba4f79a507c15eba850ceb, user_id=795dbdfdc28a4d77a4860e764496bcd2, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.224535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.228007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bad23be-6b76-4485-a37e-43073f2f35ee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.233548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.234211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.235171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.235493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.236302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.238618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.244163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 926/1868] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:08 2022] PUT /identity/v3/projects/5f6fe842ddba4f79a507c15eba850ceb/users/795dbdfdc28a4d77a4860e764496bcd2/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:08.248620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.274449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 943/1869] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:08 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:08.278342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.335962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c330934-ea71-4069-8319-cff1d6fafa23 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.350297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:08.394649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f874755-db68-404f-8995-0baf8eedf446 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.406388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.409051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.410014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.410901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.411146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.411891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.412459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.412985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.413471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.413911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.414624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.417175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.429802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 927/1870] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:08.433924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.447553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 944/1871] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:08 2022] POST /identity/v3/users => generated 301 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:08.448410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849e57c-aad2-4be8-bd7a-c9f3cfd6c0ab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.451526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.454937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51677925-06c5-4663-9ae5-91cab9567375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.456175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677925-06c5-4663-9ae5-91cab9567375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.456723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677925-06c5-4663-9ae5-91cab9567375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.457001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51677925-06c5-4663-9ae5-91cab9567375 admin admin] PATH_INFO: `/v3/projects/5f6fe842ddba4f79a507c15eba850ceb/users/795dbdfdc28a4d77a4860e764496bcd2/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.460371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51677925-06c5-4663-9ae5-91cab9567375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6fe842ddba4f79a507c15eba850ceb, user_id=795dbdfdc28a4d77a4860e764496bcd2, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.464111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51677925-06c5-4663-9ae5-91cab9567375 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.464262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfa9353-fc8f-41fa-b050-37c6bb027312 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.470444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.471166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.472375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.472492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.473182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.475333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.478253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 928/1872] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:08 2022] PUT /identity/v3/projects/5f6fe842ddba4f79a507c15eba850ceb/users/795dbdfdc28a4d77a4860e764496bcd2/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:08.490275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 945/1873] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:08.491553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.491849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.492258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.494531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.508985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:08.515951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a232e96-2d5b-4564-b029-c25193143d19 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.529163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] MFA Rules not processed for user `795dbdfdc28a4d77a4860e764496bcd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:08.530479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.612105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.612793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.613077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.613354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.614184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.616852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.647917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 946/1874] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:08.653060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.656570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.687571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 929/1875] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:08.687571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.691909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0931e4-8444-423e-b912-360ad787cc40 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.696406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494beef6-ac4f-4aa0-bfe6-830feae6e04f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.698075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.699189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.699509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.700505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] PATH_INFO: `/v3/projects/e93899145aaa458ea5b2eae78407e0cd` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.701298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93899145aaa458ea5b2eae78407e0cd)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.702163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.703231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.703459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.703776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.704161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] PATH_INFO: `/v3/projects/8c0b8c0e1dea4e248f3941e65859bcc8/users/d8386eed500e4b61888bdb7a9767f0cd/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.752122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0b8c0e1dea4e248f3941e65859bcc8, user_id=d8386eed500e4b61888bdb7a9767f0cd, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:08.761711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:08.806186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 930/1876] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:08 2022] PUT /identity/v3/projects/8c0b8c0e1dea4e248f3941e65859bcc8/users/d8386eed500e4b61888bdb7a9767f0cd/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:08.806186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:08.811996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.826518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93899145aaa458ea5b2eae78407e0cd', 'request_id': 'req-2bd87999-57d0-4a93-be9b-560c4b56235d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.845964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93899145aaa458ea5b2eae78407e0cd', 'request_id': 'req-2bd87999-57d0-4a93-be9b-560c4b56235d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.850741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93899145aaa458ea5b2eae78407e0cd', 'request_id': 'req-2bd87999-57d0-4a93-be9b-560c4b56235d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.886813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] The token cache is being invalidate because project e93899145aaa458ea5b2eae78407e0cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:08.887382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.888456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2bd87999-57d0-4a93-be9b-560c4b56235d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:08.893708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 947/1877] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:08 2022] DELETE /identity/v3/projects/e93899145aaa458ea5b2eae78407e0cd => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:08.895576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78bac-0e9f-42d2-a59d-8e03aafa1078 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.895807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78bac-0e9f-42d2-a59d-8e03aafa1078 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.898257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78bac-0e9f-42d2-a59d-8e03aafa1078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.929554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1606b7a1-aaf9-44b7-ac70-dcc04d17e5e3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:08.935595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-40f78bac-0e9f-42d2-a59d-8e03aafa1078 None None] MFA Rules not processed for user `d8386eed500e4b61888bdb7a9767f0cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:08.937447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-40f78bac-0e9f-42d2-a59d-8e03aafa1078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:08.963209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:08.989412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:08.990059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:08.990343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:08.990618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:08.992439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:09.005010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 948/1878] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:08 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:09.006225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60add-ef0b-40bc-8904-1c5ccee2f567 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.006583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60add-ef0b-40bc-8904-1c5ccee2f567 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.006939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df60add-ef0b-40bc-8904-1c5ccee2f567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.025410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0df60add-ef0b-40bc-8904-1c5ccee2f567 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:09.026614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0df60add-ef0b-40bc-8904-1c5ccee2f567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:09.031662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:09.057979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.059527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.070382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 931/1879] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:09.071188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.072592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:09.091624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 949/1880] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:09.096580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.096870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.097149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.117866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:09.119838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:09.134237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 950/1881] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:09.136571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.150979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45a897-e1ad-49db-83ac-7f8951a527f1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.156946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f002d18c-b2af-4f12-9a5b-db0f6241c800 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.158251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.159389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.159891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.160417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.162831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:09.163328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.165349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.176096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.178204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.178602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.178985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.180867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.184732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.186095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 951/1882] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:09.187054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.200184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c94e43-4211-4f05-aed2-0a6766b869a7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.206463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.207317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.207616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.207899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.208887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.210838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.235849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 932/1883] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:09 2022] POST /identity/v3/projects => generated 399 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:09.245353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.256851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 952/1884] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:09 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:09.266271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.268097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:09.364334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686221d6-b57d-424e-95a6-9321c02a0284 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.368459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.369888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.370185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.370465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.372945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:09.412677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7558b1-c351-4fbe-8ee3-06f88afa01b6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.657132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.657846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.658332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.658656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.660752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.663096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.685716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:09.708337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 933/1885] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:09 2022] POST /identity/v3/users => generated 306 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:09.709021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.714710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:09.730313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06da4a2-af9d-464f-8e7f-68e2ef6a510d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.745820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.746638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.747640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.748977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.750013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.751575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.754056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.755311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.786638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 953/1886] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:09.788135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.801145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e3a242-e1ca-482c-8a90-c78133a14695 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.807794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.809989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.811119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.811119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.812742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.815805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.836161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 954/1887] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:09.843747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 934/1888] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:09 2022] POST /identity/v3/users => generated 292 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:09.844474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.869515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9425381d-61b5-4b8b-9471-10d77f8715a4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.869705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.883745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.883745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.883745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.883745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] PATH_INFO: `/v3/projects/d29a5a9ef3244089b809fe6431380056/users/e5537107d747417ba95218241da9fd5b/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.889584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964725f5-664e-446f-bb9f-cce8a2ced3df admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:09.895792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:09.896993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:09.896993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:09.897165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:09.897899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.907556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29a5a9ef3244089b809fe6431380056, user_id=e5537107d747417ba95218241da9fd5b, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:09.908320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.910559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:09.921359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 935/1889] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:09.926145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.939416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 955/1890] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:09 2022] PUT /identity/v3/projects/d29a5a9ef3244089b809fe6431380056/users/e5537107d747417ba95218241da9fd5b/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:09.943031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:09.966525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e41692-935a-4eaa-bfc1-dddad6bac557 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.078890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89157e9-095c-422b-a4bb-a3bd37f982ee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.090118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.092995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.093297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.093583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.096752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.104780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.116745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.117441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.117730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.118013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.118750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.120981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.147317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 936/1891] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:10.152044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:10.163602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 956/1892] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:10.165550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:10.179972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3465c414-40f9-4206-9f7a-23005b61a9a0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.191096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e7b3bc-447f-4031-80f4-ade007a8fbc6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.193612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c1ae3-2427-43b9-adaa-454b1573f9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.194572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c1ae3-2427-43b9-adaa-454b1573f9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.194865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c1ae3-2427-43b9-adaa-454b1573f9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.195165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c1ae3-2427-43b9-adaa-454b1573f9bc admin admin] PATH_INFO: `/v3/projects/1844119f38ce4889b2ccf1c55c415beb/users/6b15b828b20f4dd79331d0757aec5b3a/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.211093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c495281-bbb2-467f-8e00-5a0c7ab04951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.212035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c495281-bbb2-467f-8e00-5a0c7ab04951 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.212327 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c495281-bbb2-467f-8e00-5a0c7ab04951 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.213554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c495281-bbb2-467f-8e00-5a0c7ab04951 admin admin] PATH_INFO: `/v3/projects/d29a5a9ef3244089b809fe6431380056/users/e5537107d747417ba95218241da9fd5b/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.215179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c495281-bbb2-467f-8e00-5a0c7ab04951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d29a5a9ef3244089b809fe6431380056, user_id=e5537107d747417ba95218241da9fd5b, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.218247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c495281-bbb2-467f-8e00-5a0c7ab04951 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.237887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c1ae3-2427-43b9-adaa-454b1573f9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=1844119f38ce4889b2ccf1c55c415beb, user_id=6b15b828b20f4dd79331d0757aec5b3a, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.240614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4c1ae3-2427-43b9-adaa-454b1573f9bc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.252503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 957/1893] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:10 2022] PUT /identity/v3/projects/d29a5a9ef3244089b809fe6431380056/users/e5537107d747417ba95218241da9fd5b/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:10.264361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.264677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.265015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.276791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 937/1894] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:10 2022] PUT /identity/v3/projects/1844119f38ce4889b2ccf1c55c415beb/users/6b15b828b20f4dd79331d0757aec5b3a/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:10.287211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.287850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.288371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.314499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] MFA Rules not processed for user `e5537107d747417ba95218241da9fd5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:10.315882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:10.338832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] MFA Rules not processed for user `6b15b828b20f4dd79331d0757aec5b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:10.341490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:10.454749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 938/1895] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:10.456391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:10.458316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:10.487192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 958/1896] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:10 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:10.488089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:10.516928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:10.563009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:10.623134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd198270-b2b6-4d63-a17c-260517b0cc68 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.633705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.634403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.635228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.635585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.638354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26cbfd-452a-4079-9a6b-f4f870a0c9a8 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.652161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.653363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.653653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.653932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] PATH_INFO: `/v3/users/1e4e470e9fe34059913a6acf84a05435` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.655933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e4e470e9fe34059913a6acf84a05435)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.657894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.659709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.661230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.698099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 959/1897] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:10.699286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:10.704082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:10.739354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c26e9a-53df-48d4-9a1e-4d911a3e6433 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.762681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.763405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.763702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.763982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.770670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e4e470e9fe34059913a6acf84a05435'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.780187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.780763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e4e470e9fe34059913a6acf84a05435'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.781661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.781951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e4e470e9fe34059913a6acf84a05435'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.782716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e4e470e9fe34059913a6acf84a05435'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.811693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 960/1898] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:10 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:10.811693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:10.819225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a8ce15-3c9c-4bb2-925c-10ca64042360 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:10.832443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3f8c34b-32ff-47c9-9e40-dacebee0dbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e4e470e9fe34059913a6acf84a05435'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.841546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:10.843621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.844294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.849024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] PATH_INFO: `/v3/users/6b15b828b20f4dd79331d0757aec5b3a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.851638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b15b828b20f4dd79331d0757aec5b3a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:10.854240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 939/1899] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:10 2022] DELETE /identity/v3/users/1e4e470e9fe34059913a6acf84a05435 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:10.854633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:10.859732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5832-6281-494b-b547-005c84fdf30d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:10.860026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5832-6281-494b-b547-005c84fdf30d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:10.878479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676e5832-6281-494b-b547-005c84fdf30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:10.928368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-676e5832-6281-494b-b547-005c84fdf30d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:10.929695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-676e5832-6281-494b-b547-005c84fdf30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:10.961410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b15b828b20f4dd79331d0757aec5b3a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.974532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b15b828b20f4dd79331d0757aec5b3a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.976516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b15b828b20f4dd79331d0757aec5b3a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:10.977303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b15b828b20f4dd79331d0757aec5b3a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:11.010005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b15b828b20f4dd79331d0757aec5b3a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:11.035746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 961/1900] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:10 2022] DELETE /identity/v3/users/6b15b828b20f4dd79331d0757aec5b3a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:11.035746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:11.035746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:11.101791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 940/1901] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:11.101791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:11.101791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:11.101791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:11.123479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31663497-1aff-4fc2-9363-1cb600df31cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:11.129450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:11.130389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:11.130733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:11.131028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:11.132956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:11.133103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:11.135070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:11.189219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:11.240240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:11.245427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:11.257052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 962/1902] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:11 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:11.257984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:11.260394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:11.273915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:11.320342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 941/1903] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:11.352781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:11.354710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:11.376672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090c292b-ed66-40dc-bbab-500c73cc0b28 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:11.377131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df689d-7f27-408e-ae89-548e33101114 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:11.386376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237799c-1240-400d-aa10-40db011f1119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:11.388686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237799c-1240-400d-aa10-40db011f1119 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:11.389162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237799c-1240-400d-aa10-40db011f1119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:11.389739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237799c-1240-400d-aa10-40db011f1119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:11.390422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:11.391116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:11.391433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:11.391720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:11.415581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:11.419533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:11.423017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237799c-1240-400d-aa10-40db011f1119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:11.428401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237799c-1240-400d-aa10-40db011f1119 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:11.444872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 963/1904] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:11.450934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 942/1905] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:12.364525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:12.366672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:12.398556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee63f8d-6b8d-4fa5-9eb3-e0b89cf346dd service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:12.417141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:12.418078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:12.418353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:12.418920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] PATH_INFO: `/v3/projects/036bb651fb1b4817bc8f6593293a9d2c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:12.420306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=036bb651fb1b4817bc8f6593293a9d2c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:12.422315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:12.423225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533a24-d811-4140-8e81-15eecffea9a9 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:12.424109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533a24-d811-4140-8e81-15eecffea9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:12.424641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8533a24-d811-4140-8e81-15eecffea9a9 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:12.426481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 943/1906] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:02:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 20 06:02:12.461343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:12.461654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:12.464755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:12.510674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:12.512006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:12.565964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036bb651fb1b4817bc8f6593293a9d2c', 'request_id': 'req-cb1d0174-14a7-453e-a9b4-70f31a9fe131'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.578751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036bb651fb1b4817bc8f6593293a9d2c', 'request_id': 'req-cb1d0174-14a7-453e-a9b4-70f31a9fe131'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.600025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036bb651fb1b4817bc8f6593293a9d2c', 'request_id': 'req-cb1d0174-14a7-453e-a9b4-70f31a9fe131'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.623803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] The token cache is being invalidate because project 036bb651fb1b4817bc8f6593293a9d2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:12.623803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.624586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.628998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 964/1907] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:12 2022] DELETE /identity/v3/projects/036bb651fb1b4817bc8f6593293a9d2c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:12.630848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:12.632049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:12.723585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 944/1908] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:02:12 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:12.724818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:12.726458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:12.748283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:12.760509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0622cef-ed72-46fb-9b97-f2ee3c100344 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:12.779427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:12.780489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:12.780788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:12.781070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] PATH_INFO: `/v3/projects/1844119f38ce4889b2ccf1c55c415beb` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:12.781810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d0174-14a7-453e-a9b4-70f31a9fe131 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:12.782343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1844119f38ce4889b2ccf1c55c415beb)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:12.784601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:12.788701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:12.789801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:12.790189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:12.790575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] PATH_INFO: `/v3/users/795dbdfdc28a4d77a4860e764496bcd2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:12.792553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=795dbdfdc28a4d77a4860e764496bcd2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:12.794821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:12.874367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '795dbdfdc28a4d77a4860e764496bcd2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.875126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1844119f38ce4889b2ccf1c55c415beb', 'request_id': 'req-2091793b-6e5b-4ee1-8f03-6eb3f7360229'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.881716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1844119f38ce4889b2ccf1c55c415beb', 'request_id': 'req-2091793b-6e5b-4ee1-8f03-6eb3f7360229'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.884826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '795dbdfdc28a4d77a4860e764496bcd2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.885572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '795dbdfdc28a4d77a4860e764496bcd2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.886231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '795dbdfdc28a4d77a4860e764496bcd2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.889011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1844119f38ce4889b2ccf1c55c415beb', 'request_id': 'req-2091793b-6e5b-4ee1-8f03-6eb3f7360229'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.903241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-10a0138a-78f9-4c39-a078-f85629893a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '795dbdfdc28a4d77a4860e764496bcd2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.903912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] The token cache is being invalidate because project 1844119f38ce4889b2ccf1c55c415beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:12.904199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.904843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:12.907102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 945/1909] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:12 2022] DELETE /identity/v3/projects/1844119f38ce4889b2ccf1c55c415beb => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:12.907934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:12.911419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:12.923856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 965/1910] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:12 2022] DELETE /identity/v3/users/795dbdfdc28a4d77a4860e764496bcd2 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:12.926768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629119-a614-4298-8e4a-f9743cf88a4c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:12.927073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629119-a614-4298-8e4a-f9743cf88a4c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:12.935548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b629119-a614-4298-8e4a-f9743cf88a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:12.968265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7b629119-a614-4298-8e4a-f9743cf88a4c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:12.970298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7b629119-a614-4298-8e4a-f9743cf88a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:12.976676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:13.006634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2091793b-6e5b-4ee1-8f03-6eb3f7360229 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.081952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 966/1911] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:13.084627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.084930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.085212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.127898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:13.129574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:13.144625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.146036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.146357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.146638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.149584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 967/1912] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:13.154499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:13.155296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:13.176622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0a72a-9c8d-407b-9148-8b07cc6e1aca None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.186374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.188133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.190990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.191712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.191997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.192280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.193360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:13.193682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.200362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.213694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 946/1913] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:13.234927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 968/1914] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:13.235829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:13.250380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40946d0b-4098-4908-8f21-2fa3974ee0b1 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.264622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.266305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.267850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.268832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.270639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.273728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.351387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:13.359289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 947/1915] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:13 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:13.365601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0fe59c-4fd4-42f5-864d-b8072e3eeb76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.453419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:13.470029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.471474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.472340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.473007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.473914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.476782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.558448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:13.571088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 969/1916] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:13 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:13.578917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2c2d0-845d-44d5-8182-fa09f672d256 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.585701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.585701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.585981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.586052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.586795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.589155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.602381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 948/1917] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:13.606595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:13.621478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8d924-1b15-4038-9be3-aba9d4d448bc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.627628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d79b25-2408-4682-9ab2-3fda1bda9dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.628338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d79b25-2408-4682-9ab2-3fda1bda9dec admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.629001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d79b25-2408-4682-9ab2-3fda1bda9dec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.629128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d79b25-2408-4682-9ab2-3fda1bda9dec admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.629894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d79b25-2408-4682-9ab2-3fda1bda9dec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.632125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d79b25-2408-4682-9ab2-3fda1bda9dec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.649537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 970/1918] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:13.650561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:13.667894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aaf1db-a8f6-4939-b87e-011581d365c6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:13.674333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:13.675775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.675775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.676000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] PATH_INFO: `/v3/projects/abd72a25ee474008b326b7c02ef392c8/users/ce6a0a09cbcd4e8eb044506fb75a206d/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.727524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd72a25ee474008b326b7c02ef392c8, user_id=ce6a0a09cbcd4e8eb044506fb75a206d, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:13.732231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:13.776103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 949/1919] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:13 2022] PUT /identity/v3/projects/abd72a25ee474008b326b7c02ef392c8/users/ce6a0a09cbcd4e8eb044506fb75a206d/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:13.781121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:13.781879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:13.782252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:13.819770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] MFA Rules not processed for user `ce6a0a09cbcd4e8eb044506fb75a206d`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:13.819770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:13.906430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 971/1920] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:13.910921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.002417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1357a06-f489-4e12-b424-6f7d40b9a516 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.008842 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.009762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.010260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.010630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.011848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.014423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.057903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 950/1921] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:13 2022] POST /identity/v3/projects => generated 375 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:14.061956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.075773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63befb7-669e-4721-b262-fb94aa1fd356 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.160637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.161515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.161872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.162220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.163050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.167083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.204754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 972/1922] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:14 2022] POST /identity/v3/users => generated 294 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:14.204937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.213354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.215412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.233899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536a12e7-87d9-43b7-959a-1bf2f3259c14 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.236308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:14.239957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddbf54-605a-4671-8e2d-a5aaa7da77a4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.242947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.244180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.244727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.245066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.245810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.246976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.248210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.248339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.248656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.249039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] PATH_INFO: `/v3/projects/5f6fe842ddba4f79a507c15eba850ceb` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.250595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6fe842ddba4f79a507c15eba850ceb)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.252813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.263730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 973/1923] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:14.267880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.279424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbdbd5b-8e6c-4635-9995-21feb0606ffa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.284305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.285208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.285493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.285771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] PATH_INFO: `/v3/projects/614a6fe0f86a4f4b8c926db513dfa2ef/users/91768feec4b344818e8c1b41acbd6887/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.309321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614a6fe0f86a4f4b8c926db513dfa2ef, user_id=91768feec4b344818e8c1b41acbd6887, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.310355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6fe842ddba4f79a507c15eba850ceb', 'request_id': 'req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:14.312904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.318135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6fe842ddba4f79a507c15eba850ceb', 'request_id': 'req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:14.326280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6fe842ddba4f79a507c15eba850ceb', 'request_id': 'req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:14.329120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 974/1924] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:14 2022] PUT /identity/v3/projects/614a6fe0f86a4f4b8c926db513dfa2ef/users/91768feec4b344818e8c1b41acbd6887/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:14.332755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.344197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] The token cache is being invalidate because project 5f6fe842ddba4f79a507c15eba850ceb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:14.344911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:14.345479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-efd96f58-dcc8-4a6f-8b8b-c9ea4f245116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:14.351439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 951/1925] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:14 2022] DELETE /identity/v3/projects/5f6fe842ddba4f79a507c15eba850ceb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:14.352837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00d4da-0a0a-4f4b-a172-02cdada7f1cf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.353776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00d4da-0a0a-4f4b-a172-02cdada7f1cf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.354211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00d4da-0a0a-4f4b-a172-02cdada7f1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.375597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:14.378587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8d00d4da-0a0a-4f4b-a172-02cdada7f1cf None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:14.381030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8d00d4da-0a0a-4f4b-a172-02cdada7f1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.455012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ab0d3-183e-453b-bb4c-2143f42be958 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.462916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.463649 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.463940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.464218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.464959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.466973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.471706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 952/1926] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:14.482766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 975/1927] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:14.482926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.482926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.482926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.484434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.486126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.502582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:14.503774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.506322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7534d8-392b-48c8-a586-716edc6d66d1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.516409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 953/1928] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:14.516502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3db615a-7afa-404e-8da0-3b6a0bd444f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.516502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db615a-7afa-404e-8da0-3b6a0bd444f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.516686 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db615a-7afa-404e-8da0-3b6a0bd444f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.517205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3db615a-7afa-404e-8da0-3b6a0bd444f0 admin admin] PATH_INFO: `/v3/projects/614a6fe0f86a4f4b8c926db513dfa2ef/users/91768feec4b344818e8c1b41acbd6887/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.519213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.521518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db615a-7afa-404e-8da0-3b6a0bd444f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614a6fe0f86a4f4b8c926db513dfa2ef, user_id=91768feec4b344818e8c1b41acbd6887, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.527092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3db615a-7afa-404e-8da0-3b6a0bd444f0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.530181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225d7fdf-44d9-44ab-8a32-1ba691524d00 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.538479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.539835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.541047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.541804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.550868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 976/1929] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:14 2022] PUT /identity/v3/projects/614a6fe0f86a4f4b8c926db513dfa2ef/users/91768feec4b344818e8c1b41acbd6887/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:14.551306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:14.551306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.551306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.552575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.552999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.553375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.568052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 954/1930] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:14.571733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.586293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b804f35-30ae-4ef5-8a49-c37f1234f06f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.593459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] MFA Rules not processed for user `91768feec4b344818e8c1b41acbd6887`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:14.595276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.733769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.734468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.738649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.738976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.739780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.742629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:14.751543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 977/1931] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:14.756254 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:14.765560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.767556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.810097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 955/1932] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:14 2022] POST /identity/v3/projects => generated 369 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:14.814753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:14.891668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae4bf6-e3c5-4f8d-b76b-f29cd4aef832 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.947884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:14.963104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.963798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.964081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.964354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.966489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:14.972395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47150fa5-354d-45e7-ae5b-6585e87cda53 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:14.977770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:14.979207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:14.979739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:14.980133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:14.981147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:14.983174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:15.014180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 956/1933] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:14 2022] POST /identity/v3/users => generated 285 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:15.017441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:15.030861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c4e22a-426d-472f-92ab-f952ee5b068b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:15.035585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:15.036324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:15.036609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:15.037225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:15.038013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:15.039915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:15.065261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 957/1934] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:15.070161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:15.078297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:15.080063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:15.084910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835f896-1061-406c-bb94-67c70c39a818 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:15.092578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 978/1935] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:15.098594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5f2d8-a7a2-404a-af9e-e2bb6fdaf1f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:15.099912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5f2d8-a7a2-404a-af9e-e2bb6fdaf1f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:15.099995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5f2d8-a7a2-404a-af9e-e2bb6fdaf1f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:15.100535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5f2d8-a7a2-404a-af9e-e2bb6fdaf1f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:15.101488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5f2d8-a7a2-404a-af9e-e2bb6fdaf1f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:15.103459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5f2d8-a7a2-404a-af9e-e2bb6fdaf1f9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:15.122245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 958/1936] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:15.122583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:15.136635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5a0ba-02e7-4b96-8b5d-1fe58d20133e service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:15.141832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:15.142733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:15.143015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:15.143459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] PATH_INFO: `/v3/projects/aaa8bf7b87244ac8a3c488fec16d54ca/users/5fcb569f32384f8d9d1878233d2732d6/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:15.164745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaa8bf7b87244ac8a3c488fec16d54ca, user_id=5fcb569f32384f8d9d1878233d2732d6, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:15.171534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:15.194319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 979/1937] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:15 2022] PUT /identity/v3/projects/aaa8bf7b87244ac8a3c488fec16d54ca/users/5fcb569f32384f8d9d1878233d2732d6/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:15.196533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:15.196941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:15.197190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:15.247578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:15.258251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:15.418895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 959/1938] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:15.454902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:15.460071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:15.598578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2904a-e53b-4031-bc60-1701330277f1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:15.602577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9138a37-1bc2-4042-ba47-f4d9698bb342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:15.603293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9138a37-1bc2-4042-ba47-f4d9698bb342 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:15.604140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9138a37-1bc2-4042-ba47-f4d9698bb342 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:15.604140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9138a37-1bc2-4042-ba47-f4d9698bb342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:15.616228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9138a37-1bc2-4042-ba47-f4d9698bb342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:15.617752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9138a37-1bc2-4042-ba47-f4d9698bb342 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:15.652473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 980/1939] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:16.369688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:16.372603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:16.394523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753176ad-56e7-468a-9538-d21ae5618220 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:16.402709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:16.404687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:16.404977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:16.405417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] PATH_INFO: `/v3/users/d8386eed500e4b61888bdb7a9767f0cd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:16.406956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8386eed500e4b61888bdb7a9767f0cd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:16.408934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:16.458184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:16.458716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:16.458716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:16.467392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8386eed500e4b61888bdb7a9767f0cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:16.474756 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:16.476765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8386eed500e4b61888bdb7a9767f0cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:16.477586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8386eed500e4b61888bdb7a9767f0cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:16.479592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8386eed500e4b61888bdb7a9767f0cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:16.491002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] MFA Rules not processed for user `5fcb569f32384f8d9d1878233d2732d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:16.492530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:16.496112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ac2a425f-4753-4976-be8d-bb741275cb3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8386eed500e4b61888bdb7a9767f0cd'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:16.509988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 960/1940] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:16 2022] DELETE /identity/v3/users/d8386eed500e4b61888bdb7a9767f0cd => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:16.520284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:16.520582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:16.520866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:16.546009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:16.547668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:16.636090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 981/1941] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:16 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:16.646295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 961/1942] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:16.649299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:16.651675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:16.658100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:16.662899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:16.665413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:16.818280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d699bcc8-dac8-4ecc-ac99-d93fca9158c1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:16.831551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db05bb3c-5a05-4ebe-90ad-c2ec7186c8dd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:16.835271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:16.835941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:16.836233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:16.836710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:16.857550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:16.861129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:16.881725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 962/1943] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:16.901352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:16.902514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:16.902802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:16.903545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:16.987023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:16.988577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:17.009755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 982/1944] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:16 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:17.732247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:17.733819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:17.748240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eff2a-0ba5-4fdd-a565-82378fd05ec2 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:17.754233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:17.755020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:17.755340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:17.755623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] PATH_INFO: `/v3/projects/8c0b8c0e1dea4e248f3941e65859bcc8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:17.756816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0b8c0e1dea4e248f3941e65859bcc8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:17.758843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:17.809077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0b8c0e1dea4e248f3941e65859bcc8', 'request_id': 'req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:17.815655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0b8c0e1dea4e248f3941e65859bcc8', 'request_id': 'req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:17.830759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0b8c0e1dea4e248f3941e65859bcc8', 'request_id': 'req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:17.843905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:17.851117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] The token cache is being invalidate because project 8c0b8c0e1dea4e248f3941e65859bcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:17.851926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:17.853254 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:17.855927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 963/1945] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:17 2022] DELETE /identity/v3/projects/8c0b8c0e1dea4e248f3941e65859bcc8 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:17.861382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:17.877858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624fdb2-9477-4cb0-989e-5b6f032a44b1 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:17.942155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:17.943040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:17.943363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:17.943712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] PATH_INFO: `/v3/users/e5537107d747417ba95218241da9fd5b` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:17.945003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5537107d747417ba95218241da9fd5b)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:17.947528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:17.964104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:18.008352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5537107d747417ba95218241da9fd5b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.015959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5537107d747417ba95218241da9fd5b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.017489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5537107d747417ba95218241da9fd5b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.018934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5537107d747417ba95218241da9fd5b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.036082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:18.037962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.050148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3a42c3c4-eaf1-459f-bb69-bfbccd9a400d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5537107d747417ba95218241da9fd5b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.056924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3beef0d-4b3f-452e-bdd8-58d832cbf77a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:18.062718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:18.063672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.063971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.064250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] PATH_INFO: `/v3/users/ce6a0a09cbcd4e8eb044506fb75a206d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.065617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce6a0a09cbcd4e8eb044506fb75a206d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:18.067729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:18.067848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 983/1946] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:17 2022] DELETE /identity/v3/users/e5537107d747417ba95218241da9fd5b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:18.132843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce6a0a09cbcd4e8eb044506fb75a206d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.143277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce6a0a09cbcd4e8eb044506fb75a206d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.144504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce6a0a09cbcd4e8eb044506fb75a206d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.145250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce6a0a09cbcd4e8eb044506fb75a206d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.162829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce6a0a09cbcd4e8eb044506fb75a206d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.182587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 964/1947] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:18 2022] DELETE /identity/v3/users/ce6a0a09cbcd4e8eb044506fb75a206d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:18.185413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.185862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.186566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.208487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:18.210127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.288950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 984/1948] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:18.301291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:18.302832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.379392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdab6223-97df-4067-80d5-4207ef74608d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:18.389884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c3226b-eb7a-4b00-b67a-4f1421096933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:18.390738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3226b-eb7a-4b00-b67a-4f1421096933 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.392494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3226b-eb7a-4b00-b67a-4f1421096933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.392905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c3226b-eb7a-4b00-b67a-4f1421096933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.413039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3226b-eb7a-4b00-b67a-4f1421096933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:18.414799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c3226b-eb7a-4b00-b67a-4f1421096933 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:18.446883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 965/1949] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:18.469663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:18.472033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.502312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f74e29-bbea-45ed-aec9-7d2ee69806e1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:18.519504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:18.520391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.521010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.521867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] PATH_INFO: `/v3/projects/d29a5a9ef3244089b809fe6431380056` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.525842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d29a5a9ef3244089b809fe6431380056)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:18.528628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:18.603403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd29a5a9ef3244089b809fe6431380056', 'request_id': 'req-47111302-59d6-481b-88bd-dbf0e7c9db4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.614485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd29a5a9ef3244089b809fe6431380056', 'request_id': 'req-47111302-59d6-481b-88bd-dbf0e7c9db4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.624125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd29a5a9ef3244089b809fe6431380056', 'request_id': 'req-47111302-59d6-481b-88bd-dbf0e7c9db4c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.648259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ad1aa-4ee3-4a62-9318-481697c6161e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.648684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ad1aa-4ee3-4a62-9318-481697c6161e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.649062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ad1aa-4ee3-4a62-9318-481697c6161e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.669561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] The token cache is being invalidate because project d29a5a9ef3244089b809fe6431380056 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:18.670181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.673282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47111302-59d6-481b-88bd-dbf0e7c9db4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:18.700958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 985/1950] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:18 2022] DELETE /identity/v3/projects/d29a5a9ef3244089b809fe6431380056 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:18.702490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.702793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.703051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.706188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-641ad1aa-4ee3-4a62-9318-481697c6161e None None] MFA Rules not processed for user `5fcb569f32384f8d9d1878233d2732d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:18.711643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-641ad1aa-4ee3-4a62-9318-481697c6161e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.749175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:18.750697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.775099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:18.863403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 966/1951] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:18.920679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 986/1952] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:18.929727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.930826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.931172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.960057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:18.961568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:18.972926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 967/1953] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:18.976832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:18.987980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ddf7fe-318c-48d8-8376-d6b819348540 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:18.999422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:18.999422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:18.999422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:18.999422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:18.999861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:18.999861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:18.999861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.012317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 987/1954] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:19.016845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.027932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d8529-7d08-40b7-b028-af1b056cefb0 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.033323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.035743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.036042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.036333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.037476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.039481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.081045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:19.099822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.102304 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:19.122483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 968/1955] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:19 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:19.126374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf76d54-e827-4bc8-8b35-87e1ae5b7f1c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.126663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.227833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.230410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.232501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.232749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] PATH_INFO: `/v3/projects/abd72a25ee474008b326b7c02ef392c8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.234141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd72a25ee474008b326b7c02ef392c8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.238783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.289843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108e78e7-6cd4-49e4-9cd0-e55ba9d229f6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.300279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.300984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.301321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.301585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.302434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.304645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.328953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd72a25ee474008b326b7c02ef392c8', 'request_id': 'req-20991eb5-faac-4641-bff9-b221c9b5a857'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.347077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd72a25ee474008b326b7c02ef392c8', 'request_id': 'req-20991eb5-faac-4641-bff9-b221c9b5a857'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.355134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 969/1956] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:19 2022] POST /identity/v3/users => generated 294 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:19.357283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd72a25ee474008b326b7c02ef392c8', 'request_id': 'req-20991eb5-faac-4641-bff9-b221c9b5a857'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.359824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.378170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958bf07-60d6-477c-b77a-a488d7e82f6d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.396506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] The token cache is being invalidate because project abd72a25ee474008b326b7c02ef392c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:19.396819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.398412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.403497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 988/1957] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:19 2022] DELETE /identity/v3/projects/abd72a25ee474008b326b7c02ef392c8 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:19.409560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.413629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:19.472892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.473674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.474049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.474342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.475480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.477430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.479151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:19.503108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 970/1958] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:19.504818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.506804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:19.516384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20991eb5-faac-4641-bff9-b221c9b5a857 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.522222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.523421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.523618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed93288-2e3e-4dbf-9250-a6325c192503 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.524008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.524432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] PATH_INFO: `/v3/users/91768feec4b344818e8c1b41acbd6887` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.526383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=91768feec4b344818e8c1b41acbd6887)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.528876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.529710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.531107 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.531473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.531764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.532516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.534291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.553825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 971/1959] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:19.554745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.570476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3cbb0f-77ad-499d-b4ab-f39bf616d4d3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.576598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47221690-8d9d-42fe-8d21-10c3abeb519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.577626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47221690-8d9d-42fe-8d21-10c3abeb519f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.577907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47221690-8d9d-42fe-8d21-10c3abeb519f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.578346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47221690-8d9d-42fe-8d21-10c3abeb519f admin admin] PATH_INFO: `/v3/projects/43440746de6a40d1a0c29c219851998f/users/1052c99d23a04f6cbfb168d1ee38e073/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.593263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91768feec4b344818e8c1b41acbd6887'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.601247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47221690-8d9d-42fe-8d21-10c3abeb519f admin admin] RBAC: Authorizing `identity:create_grant(project_id=43440746de6a40d1a0c29c219851998f, user_id=1052c99d23a04f6cbfb168d1ee38e073, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.602753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91768feec4b344818e8c1b41acbd6887'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.603679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91768feec4b344818e8c1b41acbd6887'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.604434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91768feec4b344818e8c1b41acbd6887'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.608941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47221690-8d9d-42fe-8d21-10c3abeb519f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.620817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91768feec4b344818e8c1b41acbd6887'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:19.633396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 972/1960] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:19 2022] PUT /identity/v3/projects/43440746de6a40d1a0c29c219851998f/users/1052c99d23a04f6cbfb168d1ee38e073/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:19.647394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.647890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.648294 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.656217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 989/1961] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:19 2022] DELETE /identity/v3/users/91768feec4b344818e8c1b41acbd6887 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:19.728549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] MFA Rules not processed for user `1052c99d23a04f6cbfb168d1ee38e073`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:19.729815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:19.779665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:19.843014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 973/1962] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:19.845798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:19.847329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:19.925554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52da967b-7fc0-4099-a8aa-edb8a427dd2c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:19.931790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:19.932584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:19.932937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:19.933470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:19.934161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:19.937736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:19.983955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 990/1963] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:19 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:19.986314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.002773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b1296-97e2-4021-802b-43034bd3a33d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.086695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.089126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:20.095587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.096335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.096867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.097157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.098073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.100718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.119330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e685253c-9181-4cdd-bd9d-43459def6a0a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.131896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 974/1964] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:19 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:20.136083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.150723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.151713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.152001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.167578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] PATH_INFO: `/v3/projects/614a6fe0f86a4f4b8c926db513dfa2ef` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.172906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614a6fe0f86a4f4b8c926db513dfa2ef)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.173291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5a0f62-9811-4781-8d75-e2ce2e3af1ca admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.180187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.186406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.187337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.188447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.188920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.190239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.192992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.207044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 975/1965] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:20.211003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.253205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31f11c-6d90-43bb-bcf4-80595038ffef admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.254949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:20.308129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614a6fe0f86a4f4b8c926db513dfa2ef', 'request_id': 'req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:20.345689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614a6fe0f86a4f4b8c926db513dfa2ef', 'request_id': 'req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:20.362586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614a6fe0f86a4f4b8c926db513dfa2ef', 'request_id': 'req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:20.389432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] The token cache is being invalidate because project 614a6fe0f86a4f4b8c926db513dfa2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:20.389844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:20.390621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.391610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.391898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.392178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] PATH_INFO: `/v3/projects/8628726ef3cb41d6a6bc1f97f4ac4f69/users/f2b9891c48324b5ea8830e87688ed825/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.393880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-948d410e-b2cd-4ca9-ae9d-6b8e62139ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:20.399070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 991/1966] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:20 2022] DELETE /identity/v3/projects/614a6fe0f86a4f4b8c926db513dfa2ef => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:20.404786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bdca2-54e5-46be-aac2-63359cb1b340 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.405739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bdca2-54e5-46be-aac2-63359cb1b340 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.405957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462bdca2-54e5-46be-aac2-63359cb1b340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.429041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8628726ef3cb41d6a6bc1f97f4ac4f69, user_id=f2b9891c48324b5ea8830e87688ed825, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.433972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.455061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-462bdca2-54e5-46be-aac2-63359cb1b340 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:20.458285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-462bdca2-54e5-46be-aac2-63359cb1b340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:20.461773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 976/1967] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:20 2022] PUT /identity/v3/projects/8628726ef3cb41d6a6bc1f97f4ac4f69/users/f2b9891c48324b5ea8830e87688ed825/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:20.465864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.482446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:20.599304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:20.663747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 992/1968] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:20.665653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.666743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.667102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.684145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634e0201-bb2f-4f32-a17c-5d6f007db6d8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.700026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.700702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:20.702039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:20.702178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.702512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.702902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.703893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.705602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.710911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 993/1969] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:20.715795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.722150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 977/1970] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:20.731921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.734674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba3f4fd-a7d6-498f-b6b9-4bfe5c9710d1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.739798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.740489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.740773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.742343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.742343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:20.742343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.742651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f0cd86-72cb-40f4-a11f-e648d362e4c3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.744452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.748151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437c40d-71b3-4899-b157-ffe2c0add4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.749125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437c40d-71b3-4899-b157-ffe2c0add4ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.749484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437c40d-71b3-4899-b157-ffe2c0add4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.749837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437c40d-71b3-4899-b157-ffe2c0add4ca admin admin] PATH_INFO: `/v3/projects/8628726ef3cb41d6a6bc1f97f4ac4f69/users/f2b9891c48324b5ea8830e87688ed825/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.751937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437c40d-71b3-4899-b157-ffe2c0add4ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=8628726ef3cb41d6a6bc1f97f4ac4f69, user_id=f2b9891c48324b5ea8830e87688ed825, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.754562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e437c40d-71b3-4899-b157-ffe2c0add4ca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.757188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 994/1971] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:20.761865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.769298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 978/1972] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:20 2022] PUT /identity/v3/projects/8628726ef3cb41d6a6bc1f97f4ac4f69/users/f2b9891c48324b5ea8830e87688ed825/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:20.773725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee05ffc5-95bd-4689-a6e4-9d2fbd89a714 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.779908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.780207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.780485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.812884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] MFA Rules not processed for user `f2b9891c48324b5ea8830e87688ed825`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:20.814374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:20.841913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.842812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.843275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.843720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.844417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.846346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.881053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 995/1973] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:20 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:20.884992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.915947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:20.916508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 979/1974] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:20.928233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.929601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:20.955646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0262ce78-258b-482d-bf5f-1d9933de62bb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:20.962804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:20.963582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:20.963849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:20.964128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:20.964877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:20.967052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:20.985985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 996/1975] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:20 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:20.990290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:20.997255 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac839883-aae5-4d33-8fc7-c08e73c32ee4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:21.001107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ddea2-c501-496b-b4d7-eaaf24ef7824 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:21.005684 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:21.006339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:21.006615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:21.006887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:21.007637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:21.009751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:21.019548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 997/1976] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:21.023460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:21.033316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feac8c6-95d5-4ecb-9471-ecba70bf26cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:21.037900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:21.038552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:21.038837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:21.039123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:21.040223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:21.041648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:21.048601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:21.049264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:21.049562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:21.049847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:21.052185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 998/1977] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:21.056301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:21.061405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:21.067271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efad5f-3aaf-4cec-949b-fbd369bc0977 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:21.072733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a544a-2359-4edf-b4ee-bfcb85e8f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:21.074274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a544a-2359-4edf-b4ee-bfcb85e8f84f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:21.074274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a544a-2359-4edf-b4ee-bfcb85e8f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:21.074487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a544a-2359-4edf-b4ee-bfcb85e8f84f admin admin] PATH_INFO: `/v3/projects/20f810b5b3a1447a9cc0bf6d5911b9a0/users/237d1a67507a43a998400d7ee4411a56/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:21.092397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a544a-2359-4edf-b4ee-bfcb85e8f84f admin admin] RBAC: Authorizing `identity:create_grant(project_id=20f810b5b3a1447a9cc0bf6d5911b9a0, user_id=237d1a67507a43a998400d7ee4411a56, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:21.096079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a544a-2359-4edf-b4ee-bfcb85e8f84f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:21.112947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 999/1978] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:21 2022] PUT /identity/v3/projects/20f810b5b3a1447a9cc0bf6d5911b9a0/users/237d1a67507a43a998400d7ee4411a56/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:21.123677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:21.123960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:21.124241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:21.170820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:21.172953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:21.199395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] MFA Rules not processed for user `237d1a67507a43a998400d7ee4411a56`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:21.200245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:21.201009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 980/1979] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:20 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:21.283852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1000/1980] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:21.295984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:21.297892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:21.310615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b0c5d-9d25-46d8-8da5-854ef5f6c9ad service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:21.369177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78de41-4824-445f-8c10-27d1d6c4777c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:21.369177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78de41-4824-445f-8c10-27d1d6c4777c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:21.369177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78de41-4824-445f-8c10-27d1d6c4777c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:21.369177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78de41-4824-445f-8c10-27d1d6c4777c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:21.383544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78de41-4824-445f-8c10-27d1d6c4777c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:21.383544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78de41-4824-445f-8c10-27d1d6c4777c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:21.391935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 981/1981] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:22.096809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:22.179892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed3bad-97a3-4e80-b0f0-00c25f12da9a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:22.185342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:22.186515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:22.186623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:22.186992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] PATH_INFO: `/v3/users/1052c99d23a04f6cbfb168d1ee38e073` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:22.189265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=1052c99d23a04f6cbfb168d1ee38e073)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:22.191616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:22.193986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:22.254792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1052c99d23a04f6cbfb168d1ee38e073'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:22.263606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1052c99d23a04f6cbfb168d1ee38e073'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:22.264407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1052c99d23a04f6cbfb168d1ee38e073'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:22.265094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1052c99d23a04f6cbfb168d1ee38e073'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:22.281087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1052c99d23a04f6cbfb168d1ee38e073'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:22.304629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1001/1982] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:22 2022] DELETE /identity/v3/users/1052c99d23a04f6cbfb168d1ee38e073 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:22.310042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:22.310529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:22.310620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:22.330362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:22.331623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:22.448157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 982/1983] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:22.458189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:22.468837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:22.611155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68daa5b-3d20-4739-be70-9089741b74ad admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:22.616786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:22.618395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:22.619208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:22.619529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:22.633130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:22.634639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:22.651606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1002/1984] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:22.992961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:22.995125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:23.014579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887bc848-7e06-40b5-b762-7a44d906f8e6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:23.021421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:23.022497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:23.023111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:23.024175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] PATH_INFO: `/v3/projects/43440746de6a40d1a0c29c219851998f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:23.026419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=43440746de6a40d1a0c29c219851998f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:23.030106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:23.113307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:23.121477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43440746de6a40d1a0c29c219851998f', 'request_id': 'req-1e4b879e-b32d-473a-8e86-e9966c9ca21d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.129585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43440746de6a40d1a0c29c219851998f', 'request_id': 'req-1e4b879e-b32d-473a-8e86-e9966c9ca21d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.137485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43440746de6a40d1a0c29c219851998f', 'request_id': 'req-1e4b879e-b32d-473a-8e86-e9966c9ca21d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.158969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] The token cache is being invalidate because project 43440746de6a40d1a0c29c219851998f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:23.159482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.160349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.164149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 983/1985] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:22 2022] DELETE /identity/v3/projects/43440746de6a40d1a0c29c219851998f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:23.168047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:23.183550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2065310d-0e6e-431c-9ec6-ae726ed38c8c service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:23.334359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:23.335158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:23.335491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:23.335768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] PATH_INFO: `/v3/users/f2b9891c48324b5ea8830e87688ed825` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:23.337223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2b9891c48324b5ea8830e87688ed825)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:23.339273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:23.376074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:23.413243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2b9891c48324b5ea8830e87688ed825'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.421333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2b9891c48324b5ea8830e87688ed825'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.422511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2b9891c48324b5ea8830e87688ed825'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.423049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2b9891c48324b5ea8830e87688ed825'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.440936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4ac1e73f-1d09-4b67-9923-e7fa874f301c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2b9891c48324b5ea8830e87688ed825'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:23.472399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1003/1986] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:23 2022] DELETE /identity/v3/users/f2b9891c48324b5ea8830e87688ed825 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:23.859537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:23.861502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:23.893132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b879e-b32d-473a-8e86-e9966c9ca21d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.006298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.007489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.007781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.008148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] PATH_INFO: `/v3/projects/8628726ef3cb41d6a6bc1f97f4ac4f69` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.011122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8628726ef3cb41d6a6bc1f97f4ac4f69)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.013429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.114585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8628726ef3cb41d6a6bc1f97f4ac4f69', 'request_id': 'req-71e16454-5459-4e8b-b8c0-358d43fff144'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:24.121101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8628726ef3cb41d6a6bc1f97f4ac4f69', 'request_id': 'req-71e16454-5459-4e8b-b8c0-358d43fff144'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:24.127594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8628726ef3cb41d6a6bc1f97f4ac4f69', 'request_id': 'req-71e16454-5459-4e8b-b8c0-358d43fff144'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:24.141440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] The token cache is being invalidate because project 8628726ef3cb41d6a6bc1f97f4ac4f69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:24.142256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:24.142378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-71e16454-5459-4e8b-b8c0-358d43fff144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:24.144091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 984/1987] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:23 2022] DELETE /identity/v3/projects/8628726ef3cb41d6a6bc1f97f4ac4f69 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:24.150507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.150804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.151083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.169405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:24.170619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:24.231022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1004/1988] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:24.239797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.240171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.240461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.258984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:24.260252 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:24.268526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 985/1989] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:24.273397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.285239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc2c92f-5795-43e4-a9b4-3e7bb32cb9a9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.289982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.291199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.291528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.291808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.292538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:24.292847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.294458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.310773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1005/1990] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:24.312019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.321216 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d461df-6830-41d7-b775-0b78aca1118a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.327419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.328086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.328380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.328652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.329343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.331238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.358949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 986/1991] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:24 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:24.363158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.373623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c418bc92-ddae-413d-8cf8-efc94657dae0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.453322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.455134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.456318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.456897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.457945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.460863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.494527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1006/1992] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:24 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:24.495313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.509864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ea93bb-dac2-4089-a846-1d39f0289bda admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.515347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.516103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.516391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.516670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.517589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.519307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.531774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 987/1993] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:24.535956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.548575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0a7c2-9c8a-4122-8f74-6f83e140d97d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.554012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302dfc79-ab8a-47ea-b18d-ccd0203d68ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.554800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dfc79-ab8a-47ea-b18d-ccd0203d68ac admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.555093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dfc79-ab8a-47ea-b18d-ccd0203d68ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.555418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302dfc79-ab8a-47ea-b18d-ccd0203d68ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.556237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302dfc79-ab8a-47ea-b18d-ccd0203d68ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.558126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302dfc79-ab8a-47ea-b18d-ccd0203d68ac admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.578568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1007/1994] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:24.582920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.602215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93815312-b959-4316-a129-479f9d079b80 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.609134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.610448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.610811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.611224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] PATH_INFO: `/v3/projects/5b8ab123a413486abeaf63a2abaf75c8/users/db22c74051a448d99702a42e906544e2/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.639337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b8ab123a413486abeaf63a2abaf75c8, user_id=db22c74051a448d99702a42e906544e2, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:24.645914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:24.676823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:24.685640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 988/1995] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:24 2022] PUT /identity/v3/projects/5b8ab123a413486abeaf63a2abaf75c8/users/db22c74051a448d99702a42e906544e2/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:24.697467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.697762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.698054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.714726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:24.746344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] MFA Rules not processed for user `db22c74051a448d99702a42e906544e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:24.749845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:24.880700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1008/1996] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:24.886002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:24.987570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6248041-7b73-4d7c-8694-c3ed14b37108 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:24.994600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:24.996376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:24.996775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:24.998008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:24.999013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.002459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.048730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 989/1997] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:24 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:25.050841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.071278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c4832-cb4c-48df-b32a-7a3b1821cd5c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.152908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.154017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.154458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.154811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.156364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.158807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.167716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.169900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.206253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a872cb6e-174b-44f4-ad95-9343d5986d81 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.212813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1009/1998] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:25 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:25.218792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.227004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.228920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.229283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.229653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] PATH_INFO: `/v3/users/237d1a67507a43a998400d7ee4411a56` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.231724 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=237d1a67507a43a998400d7ee4411a56)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.233542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.242472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe28e46-eb62-4f7e-ad34-5d7fce919eeb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.256450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.257790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.258233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.258663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.261057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.263338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.281001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1010/1999] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:25.289822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.312931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8095eb-60b3-42f4-9ee9-87cf888b3ef4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.324757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '237d1a67507a43a998400d7ee4411a56'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.327341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.331905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.332018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.332220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] PATH_INFO: `/v3/projects/9c826bf7335e461c803b3f9da89ebe52/users/630027fd849947538474947ae7ae8210/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.334952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '237d1a67507a43a998400d7ee4411a56'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.335852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '237d1a67507a43a998400d7ee4411a56'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.336683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '237d1a67507a43a998400d7ee4411a56'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.356264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-72456e29-fc14-4290-bf14-a1a8843d3c01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '237d1a67507a43a998400d7ee4411a56'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.381619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 990/2000] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:25 2022] DELETE /identity/v3/users/237d1a67507a43a998400d7ee4411a56 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:25.400330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.400897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.401473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.402626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c826bf7335e461c803b3f9da89ebe52, user_id=630027fd849947538474947ae7ae8210, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.414417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.437025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1011/2001] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:25 2022] PUT /identity/v3/projects/9c826bf7335e461c803b3f9da89ebe52/users/630027fd849947538474947ae7ae8210/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:25.442324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.444218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.446291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:25.447921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.538627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:25.563851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 991/2002] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:25.564835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.566199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.567123 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:25.585178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd85a30-1c3b-4074-807c-60cea7ffde2f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.586607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbfaf43-239b-45c9-9536-8846fe506d20 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.593079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.594557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.594949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.595363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.596635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.598463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.601249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.602832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.603127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.603954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] PATH_INFO: `/v3/users/40d24282ecb94732b39cafe72820590b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.606495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40d24282ecb94732b39cafe72820590b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.608488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.612063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1012/2003] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:25.613171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.620182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.713004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40d24282ecb94732b39cafe72820590b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.723090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e218df-7515-473b-83f4-5120c5f63048 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.726286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40d24282ecb94732b39cafe72820590b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.728621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.729749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.730042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.730328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.737569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40d24282ecb94732b39cafe72820590b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.741933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.742165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40d24282ecb94732b39cafe72820590b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.743810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.757872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1013/2004] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:25.760729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.773364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5b7ed-1eac-4d47-b1ad-76fea3d92211 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:25.784109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5d444e72-a50b-4fd0-8c15-a90f2da4cfd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40d24282ecb94732b39cafe72820590b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:25.787144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518928b2-ff20-4466-87f0-ccfa2f82b34e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:25.788908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518928b2-ff20-4466-87f0-ccfa2f82b34e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.788908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518928b2-ff20-4466-87f0-ccfa2f82b34e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.788908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518928b2-ff20-4466-87f0-ccfa2f82b34e admin admin] PATH_INFO: `/v3/projects/9c826bf7335e461c803b3f9da89ebe52/users/630027fd849947538474947ae7ae8210/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.793725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518928b2-ff20-4466-87f0-ccfa2f82b34e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c826bf7335e461c803b3f9da89ebe52, user_id=630027fd849947538474947ae7ae8210, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:25.798463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518928b2-ff20-4466-87f0-ccfa2f82b34e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:25.812544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 992/2005] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:25 2022] DELETE /identity/v3/users/40d24282ecb94732b39cafe72820590b => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:25.823509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.824061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.824526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.825340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1014/2006] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:25 2022] PUT /identity/v3/projects/9c826bf7335e461c803b3f9da89ebe52/users/630027fd849947538474947ae7ae8210/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:25.835868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:25.836090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:25.836376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:25.855191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:25.856553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.872036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] MFA Rules not processed for user `630027fd849947538474947ae7ae8210`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:25.873273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:25.954587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 993/2007] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:25.964416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1015/2008] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:25.974733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:25.976734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:26.005957 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:26.007827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:26.079701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4caf1d-0627-4211-813b-1b362ae8470d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:26.084981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:26.093434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:26.093827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:26.094938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:26.096411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37f20bf-3377-4d82-9210-7bf60280228e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:26.099453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:26.105075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:26.106384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:26.106658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:26.106947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:26.126513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:26.128376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:26.137917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:26.149345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1016/2009] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:26.192932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:26.194562 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddbe42f-97a7-43b8-a823-bd49771a6892 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:26.208036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 994/2010] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:26.599133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:26.603481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:26.621028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:26.642357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ae516d-aa08-4567-b4b7-44142c5058d2 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:26.649376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:26.652076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:26.652493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:26.652841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] PATH_INFO: `/v3/projects/20f810b5b3a1447a9cc0bf6d5911b9a0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:26.654438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20f810b5b3a1447a9cc0bf6d5911b9a0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:26.656689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:26.755533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20f810b5b3a1447a9cc0bf6d5911b9a0', 'request_id': 'req-e286a591-9968-464d-96d4-b366c6a19471'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:26.776364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20f810b5b3a1447a9cc0bf6d5911b9a0', 'request_id': 'req-e286a591-9968-464d-96d4-b366c6a19471'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:26.790674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20f810b5b3a1447a9cc0bf6d5911b9a0', 'request_id': 'req-e286a591-9968-464d-96d4-b366c6a19471'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:26.832705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] The token cache is being invalidate because project 20f810b5b3a1447a9cc0bf6d5911b9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:26.833111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:26.833801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e286a591-9968-464d-96d4-b366c6a19471 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:26.841427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:26.841691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1017/2011] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:26 2022] DELETE /identity/v3/projects/20f810b5b3a1447a9cc0bf6d5911b9a0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:26.841806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:26.842049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:26.868794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:26.870203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:26.975432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 995/2012] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:26.977344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:26.977707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:26.977995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.000593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:27.001807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:27.015269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1018/2013] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:27.016861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.027118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbf4a9c-eff4-4bcc-82ff-6b84c9704d6d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.032816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.033501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.034040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.034040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.034801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:27.035117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.036805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.055414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 996/2014] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:27.056591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.066039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b805ba-443c-45b1-a75f-c8c310a449f6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.073291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.074241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.074680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.075109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.075991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.078711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.111078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1019/2015] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:27 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:27.112366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.123333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db245f97-a673-4161-9683-2ad463801285 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.183837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.184526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.184817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.185100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.185821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.188390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.213231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 997/2016] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:27 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:27.214728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.227981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd2f91-129b-4d33-97f6-3eb0297ef849 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.234340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.235158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.235481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.235766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.236498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.238121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.249550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1020/2017] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:27.256132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.267522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797fa24-c920-4af3-bb79-c936c09733b3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.273139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a11525f-2869-4bb8-8798-e69476df8363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.273812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11525f-2869-4bb8-8798-e69476df8363 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.274099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11525f-2869-4bb8-8798-e69476df8363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.274384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a11525f-2869-4bb8-8798-e69476df8363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.276240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11525f-2869-4bb8-8798-e69476df8363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.277560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a11525f-2869-4bb8-8798-e69476df8363 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.293801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 998/2018] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:27.295038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.305576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb1f1e-4ad3-4b0c-ba60-569f12fef1b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.310708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.311737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.312096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.312452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] PATH_INFO: `/v3/projects/7ebe4d344e324bb68dec94a296d31b5a/users/1e11d00ff8724073a0c9b229cb44167f/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.329618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ebe4d344e324bb68dec94a296d31b5a, user_id=1e11d00ff8724073a0c9b229cb44167f, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.332390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.348774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1021/2019] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:27 2022] PUT /identity/v3/projects/7ebe4d344e324bb68dec94a296d31b5a/users/1e11d00ff8724073a0c9b229cb44167f/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:27.360282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.361196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.361807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.395859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] MFA Rules not processed for user `1e11d00ff8724073a0c9b229cb44167f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:27.398150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:27.426193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:27.442868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.453209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:27.494033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 999/2020] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:27.507051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.510178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:27.576802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c58b9d2-b62f-4714-815d-919030d57d3d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.588417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.589307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.589594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.589880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] PATH_INFO: `/v3/projects/5b9e254898004b09b4cb885636734787` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.592181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b9e254898004b09b4cb885636734787)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.595793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.600864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c422b2-f56b-407c-b581-f1d4b44c0559 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.606439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.607113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.607444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.607732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.617993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.620110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.630491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1000/2021] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:27.640509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:27.668318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b9e254898004b09b4cb885636734787', 'request_id': 'req-4c241865-4d98-48d8-b27a-d6cec3ae14d5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.675301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b9e254898004b09b4cb885636734787', 'request_id': 'req-4c241865-4d98-48d8-b27a-d6cec3ae14d5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.684214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b9e254898004b09b4cb885636734787', 'request_id': 'req-4c241865-4d98-48d8-b27a-d6cec3ae14d5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.706569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] The token cache is being invalidate because project 5b9e254898004b09b4cb885636734787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:27.706945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.708482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.713354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1022/2022] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:27 2022] DELETE /identity/v3/projects/5b9e254898004b09b4cb885636734787 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:27.714732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:27.729002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37127d5-918b-4698-bd6b-cf74f1a99c29 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:27.812342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:27.813496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:27.813805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:27.814082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] PATH_INFO: `/v3/users/81a196eedb864611b291c3d3bd47b5f1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:27.815376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a196eedb864611b291c3d3bd47b5f1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:27.817621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:27.882271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a196eedb864611b291c3d3bd47b5f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.894313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a196eedb864611b291c3d3bd47b5f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.895335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a196eedb864611b291c3d3bd47b5f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.901733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a196eedb864611b291c3d3bd47b5f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:27.911703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:27.925561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b0e0e0b3-bb9b-47f3-85c2-b4f6f5489550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a196eedb864611b291c3d3bd47b5f1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:28.001893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1001/2023] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:27 2022] DELETE /identity/v3/users/81a196eedb864611b291c3d3bd47b5f1 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:28.380234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:28.381812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:28.408561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c241865-4d98-48d8-b27a-d6cec3ae14d5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:28.477512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:28.479295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.479625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.479941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] PATH_INFO: `/v3/projects/ee5cb432ab4644af99a5123a6586ca55` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.481484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5cb432ab4644af99a5123a6586ca55)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:28.483607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:28.534504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5cb432ab4644af99a5123a6586ca55', 'request_id': 'req-ee41592c-4dc2-456d-8447-2fe0810a061d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:28.540043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5cb432ab4644af99a5123a6586ca55', 'request_id': 'req-ee41592c-4dc2-456d-8447-2fe0810a061d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:28.546579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5cb432ab4644af99a5123a6586ca55', 'request_id': 'req-ee41592c-4dc2-456d-8447-2fe0810a061d'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:28.559417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] The token cache is being invalidate because project ee5cb432ab4644af99a5123a6586ca55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:28.559709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:28.560374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ee41592c-4dc2-456d-8447-2fe0810a061d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:28.561942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1023/2024] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:28 2022] DELETE /identity/v3/projects/ee5cb432ab4644af99a5123a6586ca55 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:28.566977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.567306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.567593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.585180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:28.586393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:28.647982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1002/2025] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:28.656154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.656695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.656695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.677311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:28.678542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:28.687325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1024/2026] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:28.691469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:28.701487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47069b5-b9f6-4cfb-8a36-16f57508904a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:28.706727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:28.707732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.708110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.708401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.709145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:28.709460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:28.711071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:28.723589 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1003/2027] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:28.728920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:28.738895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8661b25f-ed67-4786-8aa8-fb5f64a6190a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:28.743824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:28.744490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.745097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.745097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.745749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:28.747705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:28.782047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1025/2028] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:28 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:28.791800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:28.802889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a07ea1f-8097-49f9-9c71-168e6e215549 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:28.882006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:28.895213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.895213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.895213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.895654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:28.897792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:28.933615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:28.934185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1004/2029] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:28 2022] POST /identity/v3/users => generated 282 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:28.946192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a92f889-7559-4703-ab28-19e33b387bef admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:28.951456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:28.952259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:28.952615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:28.953117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:28.957846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:28.959758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:28.977065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1026/2030] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:28.982068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.000397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2cde41-5600-4472-a599-496e37201c05 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.006461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3455ea-474c-4478-9bbf-e3abf741085e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.007665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3455ea-474c-4478-9bbf-e3abf741085e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.007762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3455ea-474c-4478-9bbf-e3abf741085e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.008233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3455ea-474c-4478-9bbf-e3abf741085e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.009349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3455ea-474c-4478-9bbf-e3abf741085e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.011282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3455ea-474c-4478-9bbf-e3abf741085e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.052435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1005/2031] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:29.052750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.067511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:29.071287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd435a-f49a-4a82-ab54-5f79ae4a957a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.086645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b64b5-b3c9-4a41-9d80-625e55a15eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.092520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b64b5-b3c9-4a41-9d80-625e55a15eec admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.092844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b64b5-b3c9-4a41-9d80-625e55a15eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.093251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b64b5-b3c9-4a41-9d80-625e55a15eec admin admin] PATH_INFO: `/v3/projects/dd98cf91b8fa473eb8ae9de60b6e7dbc/users/9b6705b131aa494b8fd10561a335e5b3/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.126599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.126899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.127180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.133394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b64b5-b3c9-4a41-9d80-625e55a15eec admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd98cf91b8fa473eb8ae9de60b6e7dbc, user_id=9b6705b131aa494b8fd10561a335e5b3, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.137652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b64b5-b3c9-4a41-9d80-625e55a15eec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.156443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1027/2032] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:29 2022] PUT /identity/v3/projects/dd98cf91b8fa473eb8ae9de60b6e7dbc/users/9b6705b131aa494b8fd10561a335e5b3/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:29.168786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] MFA Rules not processed for user `1e11d00ff8724073a0c9b229cb44167f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:29.169974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.170252 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:29.170357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.170692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.210946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] MFA Rules not processed for user `9b6705b131aa494b8fd10561a335e5b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:29.213100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:29.257076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:29.281281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1006/2033] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:29.311000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1028/2034] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:29.315849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.416699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94570-609e-40e2-a87c-c1801e0c56d4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.432232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.432923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.433610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.433894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.434769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.441662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.508742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1007/2035] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:29 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:29.512702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.527820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad8a8bb-3356-4199-8d3d-26f32bc29ea0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.598448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.599326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.600170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.600170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.600777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.604007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.632057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1029/2036] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:29 2022] POST /identity/v3/users => generated 289 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:29.635396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.648746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c981999-f679-4669-bc0e-1381b3ce58c9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.655814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.656614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.656995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.657434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.658592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.661105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.674714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1008/2037] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:29.678767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.689055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e71591-ac7b-4028-82ce-a50f2ecde39e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.694325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.695498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.695878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.696283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] PATH_INFO: `/v3/projects/53576c75726b430fbc9bcd10c15a1f0a/users/cc1e66ce594f4150b96d60cfc90ffdf1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.713285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53576c75726b430fbc9bcd10c15a1f0a, user_id=cc1e66ce594f4150b96d60cfc90ffdf1, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.717076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.737866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1030/2038] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:29 2022] PUT /identity/v3/projects/53576c75726b430fbc9bcd10c15a1f0a/users/cc1e66ce594f4150b96d60cfc90ffdf1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:29.741478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.759382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df90bc65-044a-4c39-ab5f-297229b46d3c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.850292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce945ba-6023-4afc-80be-222de086395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.851143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce945ba-6023-4afc-80be-222de086395b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.851786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce945ba-6023-4afc-80be-222de086395b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.852141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce945ba-6023-4afc-80be-222de086395b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.852993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce945ba-6023-4afc-80be-222de086395b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.855082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce945ba-6023-4afc-80be-222de086395b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.874109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1009/2039] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:29.876438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:29.888776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d55d8-d9d7-448f-9322-b57cadfbfe00 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:29.894224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:29.895191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.895586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.896313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] PATH_INFO: `/v3/projects/53576c75726b430fbc9bcd10c15a1f0a/users/cc1e66ce594f4150b96d60cfc90ffdf1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.898843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53576c75726b430fbc9bcd10c15a1f0a, user_id=cc1e66ce594f4150b96d60cfc90ffdf1, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:29.901706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:29.918461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1031/2040] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:29 2022] PUT /identity/v3/projects/53576c75726b430fbc9bcd10c15a1f0a/users/cc1e66ce594f4150b96d60cfc90ffdf1/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:29.929265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:29.929668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:29.929969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:29.960738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] MFA Rules not processed for user `cc1e66ce594f4150b96d60cfc90ffdf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:29.962708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:30.028466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:30.061332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1010/2041] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:29 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:30.073132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:30.074964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:30.129000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:30.162066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071677e-a915-4251-b6c0-133d834c31f3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:30.166267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:30.166922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:30.167208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:30.167543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:30.177256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:30.179224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:30.190602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1032/2042] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:30.261732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:30.271870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1bb9b-92a4-4b6e-ba22-62d1cb677d38 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:30.275852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:30.276586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:30.276907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:30.277188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:30.337740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:30.339215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:30.348635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1011/2043] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:30 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:31.019117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:31.028607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a838694-f99b-4d9c-9c7f-cd8a414690e4 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:31.032206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:31.032883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:31.033172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:31.033449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:31.035227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:31.099789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:31.101226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f58f4-2ad6-4a73-a96d-df82d339a302 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:31.111704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1033/2044] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:33.126885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:33.191153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ccb71-03fa-49f1-99e8-9b792e4470ed service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:33.196815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:33.197833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:33.198120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:33.198400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] PATH_INFO: `/v3/users/9b6705b131aa494b8fd10561a335e5b3` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:33.200000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b6705b131aa494b8fd10561a335e5b3)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:33.202084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:33.258409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b6705b131aa494b8fd10561a335e5b3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:33.266331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b6705b131aa494b8fd10561a335e5b3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:33.267045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b6705b131aa494b8fd10561a335e5b3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:33.267700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b6705b131aa494b8fd10561a335e5b3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:33.279974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b6705b131aa494b8fd10561a335e5b3'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:33.300202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1012/2045] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:33 2022] DELETE /identity/v3/users/9b6705b131aa494b8fd10561a335e5b3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:33.307430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:33.309029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:33.309629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:33.330503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:33.332268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:33.439766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1034/2046] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:33.450429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:33.453744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:33.462476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:33.525927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddfce35-d7f3-477b-b22e-4417fe3184dd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:33.529621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:33.530286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:33.530578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:33.530862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:33.541345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:33.543211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:33.553992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1013/2047] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:33.990912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:33.992422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:34.004432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc9f5d-c34d-4e47-a098-da23017fd8fe None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:34.009896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:34.010812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:34.011056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:34.011366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] PATH_INFO: `/v3/projects/dd98cf91b8fa473eb8ae9de60b6e7dbc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:34.012791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd98cf91b8fa473eb8ae9de60b6e7dbc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:34.014710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:34.060993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd98cf91b8fa473eb8ae9de60b6e7dbc', 'request_id': 'req-420dee1a-aad4-4759-97ae-bff86a990f94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.064976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:34.067282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd98cf91b8fa473eb8ae9de60b6e7dbc', 'request_id': 'req-420dee1a-aad4-4759-97ae-bff86a990f94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.076089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd98cf91b8fa473eb8ae9de60b6e7dbc', 'request_id': 'req-420dee1a-aad4-4759-97ae-bff86a990f94'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.088187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] The token cache is being invalidate because project dd98cf91b8fa473eb8ae9de60b6e7dbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:34.088475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.089090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.090640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1035/2048] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:33 2022] DELETE /identity/v3/projects/dd98cf91b8fa473eb8ae9de60b6e7dbc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:34.094993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:34.105002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9d466-d4e2-4f6a-9a81-4efbe058af0e service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:34.162648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:34.163723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:34.164012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:34.164292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] PATH_INFO: `/v3/users/cc1e66ce594f4150b96d60cfc90ffdf1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:34.166077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1e66ce594f4150b96d60cfc90ffdf1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:34.168043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:34.213856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1e66ce594f4150b96d60cfc90ffdf1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.221414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1e66ce594f4150b96d60cfc90ffdf1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.222524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1e66ce594f4150b96d60cfc90ffdf1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.222960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1e66ce594f4150b96d60cfc90ffdf1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.235865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-87b4b807-1407-4922-95f4-4648c5d2d035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1e66ce594f4150b96d60cfc90ffdf1'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.249689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1014/2049] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:34 2022] DELETE /identity/v3/users/cc1e66ce594f4150b96d60cfc90ffdf1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:34.535494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:34.537080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:34.585955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420dee1a-aad4-4759-97ae-bff86a990f94 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:34.673375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:34.674659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:34.675101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:34.675575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] PATH_INFO: `/v3/projects/53576c75726b430fbc9bcd10c15a1f0a` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:34.677321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53576c75726b430fbc9bcd10c15a1f0a)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:34.679538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:34.734214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53576c75726b430fbc9bcd10c15a1f0a', 'request_id': 'req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.739603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53576c75726b430fbc9bcd10c15a1f0a', 'request_id': 'req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.746282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53576c75726b430fbc9bcd10c15a1f0a', 'request_id': 'req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.762187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] The token cache is being invalidate because project 53576c75726b430fbc9bcd10c15a1f0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:34.762535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.763444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6eba1bb4-b95a-4f36-ae67-6ffca78a56d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:34.765424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1036/2050] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:34 2022] DELETE /identity/v3/projects/53576c75726b430fbc9bcd10c15a1f0a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:34.771083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:34.771997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:34.772359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:34.803820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:34.805036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:34.833264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:34.879719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1015/2051] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:34.887618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:34.887973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:34.888194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:34.910303 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:34.912061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:34.921387 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1037/2052] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:34.925487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:34.954448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97230c98-f495-40c1-904b-69bf1ff39b8d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:34.962624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:34.964205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:34.964473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:34.964857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:34.965846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:34.966512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:34.970170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.011496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.019457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1016/2053] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:35.023683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d8cf7b-0310-497a-9907-464bd94dd480 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.032028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.032891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.033191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.033475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.034834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.037039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.081126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.093009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1038/2054] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:35 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:35.099293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f18fe9-0fb0-41f3-b692-6a05ea1247cc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.228980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.230360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.230799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.231305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.232181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.234317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.264132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1017/2055] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:35 2022] POST /identity/v3/users => generated 293 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:35.267583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.281022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591df0d5-032e-4d53-8765-0b78a32bbaf5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.293917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.295497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.295879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.296572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.297418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.300199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.314473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1039/2056] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:35.318972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.329434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fbc1a-4d96-453f-b27c-be78f3aa24a1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.334322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e973593-6607-4938-865c-b84c4fd0f10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.335187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e973593-6607-4938-865c-b84c4fd0f10a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.335510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e973593-6607-4938-865c-b84c4fd0f10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.335837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e973593-6607-4938-865c-b84c4fd0f10a admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.336614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e973593-6607-4938-865c-b84c4fd0f10a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.338184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e973593-6607-4938-865c-b84c4fd0f10a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.349560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1018/2057] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:35.354498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.364986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d6d3c2-b16c-4535-87f8-eb8dc2eb57da admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.371508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.372564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.372856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.373158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] PATH_INFO: `/v3/projects/a396fdf121ea4308a2efe49416bd1c96/users/f435937946eb47c0bab7e0e6b5d48f2d/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.391422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a396fdf121ea4308a2efe49416bd1c96, user_id=f435937946eb47c0bab7e0e6b5d48f2d, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.394116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.408929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1040/2058] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:35 2022] PUT /identity/v3/projects/a396fdf121ea4308a2efe49416bd1c96/users/f435937946eb47c0bab7e0e6b5d48f2d/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:35.421742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.422057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.422340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.453989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] MFA Rules not processed for user `f435937946eb47c0bab7e0e6b5d48f2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:35.455244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:35.534031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1019/2059] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:35.538504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.550913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:35.609220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e6cf7-6f3a-4cff-b0bb-af7e75e2e477 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.614271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.614999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.615230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.615627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.616664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.619748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.652580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1041/2060] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:35 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:35.658653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.688490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bf5b17-c471-43be-8e2c-6068bef101cf None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.731597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:35.825189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.825868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.826162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.826446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.827162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.829333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.875055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1020/2061] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:35 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:35.876667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.891452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a378fadb-66ec-45e8-8898-9b1095de9cf3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.897902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.898906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.899430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.899876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.900809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.903381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.929785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1042/2062] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:35.934812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.948680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16009-b43c-4690-aaa4-b65660386788 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.957793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a760a-c33d-4e9c-8eaf-882c1b6ff97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.958694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a760a-c33d-4e9c-8eaf-882c1b6ff97e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.959159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a760a-c33d-4e9c-8eaf-882c1b6ff97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.959650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a760a-c33d-4e9c-8eaf-882c1b6ff97e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:35.960575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a760a-c33d-4e9c-8eaf-882c1b6ff97e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:35.962458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496a760a-c33d-4e9c-8eaf-882c1b6ff97e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:35.975334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1021/2063] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:35.978992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:35.989818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d09e6-fcb7-4466-b515-27d5628b3da2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:35.995579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:35.996561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:35.996854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:35.997136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] PATH_INFO: `/v3/projects/344574c939c8406cbdd1c6debe7a9668/users/f72b99b42f1b44bd826de2e84535bbaf/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.013848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344574c939c8406cbdd1c6debe7a9668, user_id=f72b99b42f1b44bd826de2e84535bbaf, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.016662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.060246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1043/2064] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:35 2022] PUT /identity/v3/projects/344574c939c8406cbdd1c6debe7a9668/users/f72b99b42f1b44bd826de2e84535bbaf/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:36.062816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.063106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.063427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.092641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] MFA Rules not processed for user `f72b99b42f1b44bd826de2e84535bbaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:36.093887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.165500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1022/2065] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:36.170113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.234329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cb0e28-56a5-4cc4-972b-b3c1958c4833 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.241155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.241155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.241155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.241155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.241521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.241958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.243969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.244218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.258756 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd793aee-fe5c-4791-82c6-266de1b87ac5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.264539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.265488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.265776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.266067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] PATH_INFO: `/v3/users/db22c74051a448d99702a42e906544e2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.267741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db22c74051a448d99702a42e906544e2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.270010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.283663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1044/2066] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:36 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:36.287376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.347121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db22c74051a448d99702a42e906544e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:36.358631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db22c74051a448d99702a42e906544e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:36.359447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db22c74051a448d99702a42e906544e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:36.360427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db22c74051a448d99702a42e906544e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:36.363143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:36.380125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4c96a3b0-27a6-49b2-b211-29378f08ed41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db22c74051a448d99702a42e906544e2'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:36.385623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b31f43c-19ee-4f8b-9d93-bd7da80e30d2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.390774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.391494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.391934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.392213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.393063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.395178 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.400955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1023/2067] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:36 2022] DELETE /identity/v3/users/db22c74051a448d99702a42e906544e2 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:36.403326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.404164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.404537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.414160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:36.419023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1045/2068] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:36 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:36.426036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.429403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.434020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:36.435623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.445438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66e1df2-9d44-4ece-bfff-da868a67d3cb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.508174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1024/2069] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:36.515122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.515846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.516130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.516410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.517254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.518480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.519639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.522461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.533875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1046/2070] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:36.537685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.547793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161b1e57-717d-4886-8b10-fa2eb30f55ed admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.552538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.553395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.553685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.553969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] PATH_INFO: `/v3/projects/ad8abf874f8c42ee99c77d6962252ab1/users/366121896c82453a8f74796c9f37ea25/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.572750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8abf874f8c42ee99c77d6962252ab1, user_id=366121896c82453a8f74796c9f37ea25, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.575705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.580696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f2169-a3a9-45d3-a6ae-37081a7351c7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.584914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.585648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.586049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.586337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.591442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1047/2071] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:36 2022] PUT /identity/v3/projects/ad8abf874f8c42ee99c77d6962252ab1/users/366121896c82453a8f74796c9f37ea25/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:36.595283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.601599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.603151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.614332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1025/2072] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:36.672841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67327918-ec44-48b6-b4aa-2368f5225cfe admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.679685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5631e7d-3118-4389-b9c0-3dc77f871165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.680544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5631e7d-3118-4389-b9c0-3dc77f871165 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.680923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5631e7d-3118-4389-b9c0-3dc77f871165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.681293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5631e7d-3118-4389-b9c0-3dc77f871165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.682153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5631e7d-3118-4389-b9c0-3dc77f871165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.683947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5631e7d-3118-4389-b9c0-3dc77f871165 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.699310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1048/2073] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:36.702238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.715198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d979f9b2-9a39-43da-aec5-77fe539cc9e4 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:36.736324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:36.737491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.738057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.738538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] PATH_INFO: `/v3/projects/ad8abf874f8c42ee99c77d6962252ab1/users/366121896c82453a8f74796c9f37ea25/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.741001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad8abf874f8c42ee99c77d6962252ab1, user_id=366121896c82453a8f74796c9f37ea25, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:36.744264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:36.757847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1026/2074] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:36 2022] PUT /identity/v3/projects/ad8abf874f8c42ee99c77d6962252ab1/users/366121896c82453a8f74796c9f37ea25/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:36.769317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:36.769785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:36.770105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:36.800987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] MFA Rules not processed for user `366121896c82453a8f74796c9f37ea25`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:36.802546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.853698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:36.896757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1049/2075] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:36.908180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.909540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:36.966462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:36.968062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:37.026676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931314c0-c818-44f0-b640-13f93aefccdc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:37.030658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:37.031409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:37.032295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:37.032295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:37.042409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:37.044231 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:37.058619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1027/2076] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:37.063820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135da4b-a43c-4014-a78b-2f8917b03e8c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:37.068583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:37.069516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:37.069801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:37.070287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] PATH_INFO: `/v3/projects/5b8ab123a413486abeaf63a2abaf75c8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:37.072163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b8ab123a413486abeaf63a2abaf75c8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:37.074079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:37.134296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b8ab123a413486abeaf63a2abaf75c8', 'request_id': 'req-12374b5d-07b1-4653-942b-aa77974f55b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.149027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b8ab123a413486abeaf63a2abaf75c8', 'request_id': 'req-12374b5d-07b1-4653-942b-aa77974f55b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.155536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b8ab123a413486abeaf63a2abaf75c8', 'request_id': 'req-12374b5d-07b1-4653-942b-aa77974f55b5'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.168271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] The token cache is being invalidate because project 5b8ab123a413486abeaf63a2abaf75c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:37.168584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.169236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.170824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1050/2077] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:36 2022] DELETE /identity/v3/projects/5b8ab123a413486abeaf63a2abaf75c8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:37.175021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:37.185907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbeb8c8-7464-458a-b66b-89237d3ec3df service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:37.246241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:37.277677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:37.278591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:37.278884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:37.279190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] PATH_INFO: `/v3/users/630027fd849947538474947ae7ae8210` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:37.280401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=630027fd849947538474947ae7ae8210)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:37.282311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:37.342022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '630027fd849947538474947ae7ae8210'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.351914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '630027fd849947538474947ae7ae8210'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.352713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '630027fd849947538474947ae7ae8210'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.353341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '630027fd849947538474947ae7ae8210'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.386521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '630027fd849947538474947ae7ae8210'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:37.406637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1028/2078] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:37 2022] DELETE /identity/v3/users/630027fd849947538474947ae7ae8210 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:37.682648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:37.684402 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:37.749385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12374b5d-07b1-4653-942b-aa77974f55b5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:37.767396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:37.786332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:37.788313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:37.853973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:37.854811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:37.855102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:37.855418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:37.857528 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:37.901117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55520886-7156-49a6-9b54-c9404bdbefc3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:37.915502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:37.916553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:37.916848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:37.917125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] PATH_INFO: `/v3/projects/9c826bf7335e461c803b3f9da89ebe52` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:37.933416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c826bf7335e461c803b3f9da89ebe52)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:37.935617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:37.953901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:37.955574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edefdb0-e317-4207-b779-65f75f7abd82 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:37.970696 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1051/2079] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:38.006866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c826bf7335e461c803b3f9da89ebe52', 'request_id': 'req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:38.012101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:38.015024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c826bf7335e461c803b3f9da89ebe52', 'request_id': 'req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:38.027035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c826bf7335e461c803b3f9da89ebe52', 'request_id': 'req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:38.048462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] The token cache is being invalidate because project 9c826bf7335e461c803b3f9da89ebe52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:38.048462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:38.049133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f00e9561-1d3e-4d31-bea6-7f8eb685fe59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:38.056367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1029/2080] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:37 2022] DELETE /identity/v3/projects/9c826bf7335e461c803b3f9da89ebe52 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:38.058855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.059663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.060094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.082976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:38.084823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:38.176162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1052/2081] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:38.185411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.185737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.186070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.229100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:38.229578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:38.253085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1030/2082] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:38.257571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.271719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fadac4-0ed9-4c5e-b53c-df0da6df678c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.277716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.278444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.278732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.279006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.279927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:38.280225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.281836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.297824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1053/2083] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:38.305105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.322186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2526-f2a5-48f2-a482-91db921a215e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.333168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.334491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.334796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.335506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.336748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.339612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.384249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1031/2084] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:38 2022] POST /identity/v3/projects => generated 363 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:38.385879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.415636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f5136-c032-42d4-a885-85eb2b6fdff3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.508311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:38.523373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.524217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.524405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.524715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.525453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.527390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.570093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1054/2085] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:38 2022] POST /identity/v3/users => generated 282 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:38.572425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.585726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edbbf8-1d01-4b15-ba7a-b5c5b9899802 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.596202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.597206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.597824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.598315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.599206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.602986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.617386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1032/2086] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:38.622339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.633942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f00c90-272d-421d-bffd-5f597c2c0ae4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.639809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66186b71-f02f-4074-ba2c-6ca49ec02aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.640739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66186b71-f02f-4074-ba2c-6ca49ec02aaf admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.641165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66186b71-f02f-4074-ba2c-6ca49ec02aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.641515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66186b71-f02f-4074-ba2c-6ca49ec02aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.642322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66186b71-f02f-4074-ba2c-6ca49ec02aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.644335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66186b71-f02f-4074-ba2c-6ca49ec02aaf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.657512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1055/2087] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:38.662267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.678581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcc970-de81-4a3f-a67e-4ff5d29c77ab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.685144 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.686192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.686557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.687067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] PATH_INFO: `/v3/projects/0f801e05e533408ca474ab40d649f7bf/users/9abe16bf2cf941bf89cbecb889fe1e64/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.713134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f801e05e533408ca474ab40d649f7bf, user_id=9abe16bf2cf941bf89cbecb889fe1e64, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.716239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.730100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1033/2088] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:38 2022] PUT /identity/v3/projects/0f801e05e533408ca474ab40d649f7bf/users/9abe16bf2cf941bf89cbecb889fe1e64/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:38.752764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.752764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.752764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.783030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] MFA Rules not processed for user `9abe16bf2cf941bf89cbecb889fe1e64`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:38.784458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:38.857464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1056/2089] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:38 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:38.869030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:38.870765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:38.930831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63480d4-0092-492c-89d1-6a0bd08f900b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:38.934620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d087a68-1411-43e0-9e54-b49e419ba51d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:38.935266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d087a68-1411-43e0-9e54-b49e419ba51d service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:38.935587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d087a68-1411-43e0-9e54-b49e419ba51d service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:38.935849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d087a68-1411-43e0-9e54-b49e419ba51d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:38.950161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d087a68-1411-43e0-9e54-b49e419ba51d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:38.951634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d087a68-1411-43e0-9e54-b49e419ba51d service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:38.963043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1034/2090] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:38 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:42.332057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:42.333920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:42.354155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de2e3f-a444-4bf7-9d07-f3f552c50d47 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:42.359728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:42.360931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:42.361909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:42.362682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:42.367110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:42.528595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:42.530433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:42.548718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1057/2091] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:43.718800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:43.719415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:43.719415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:43.737766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] MFA Rules not processed for user `5fcb569f32384f8d9d1878233d2732d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:43.739064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:43.808032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1035/2092] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:44.895634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:44.897288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.004541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e753b1a4-d4af-4fbc-850f-835c6dafbb30 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.012121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.013438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.013959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.014512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] PATH_INFO: `/v3/users/dfbd42cb9a0d4c1ab17f74c967067f08` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.016027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbd42cb9a0d4c1ab17f74c967067f08)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.019005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.022226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:45.082875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbd42cb9a0d4c1ab17f74c967067f08'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.096801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbd42cb9a0d4c1ab17f74c967067f08'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.097604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbd42cb9a0d4c1ab17f74c967067f08'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.098224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbd42cb9a0d4c1ab17f74c967067f08'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.113519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbd42cb9a0d4c1ab17f74c967067f08'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.129450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1058/2093] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:44 2022] DELETE /identity/v3/users/dfbd42cb9a0d4c1ab17f74c967067f08 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:45.307856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.309981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.325201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f375f198-2393-4eaf-8ae0-68f15b4d2754 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.337766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.339196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.356837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:45.394074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.394881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.395166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.395472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] PATH_INFO: `/v3/users/f435937946eb47c0bab7e0e6b5d48f2d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.396666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f435937946eb47c0bab7e0e6b5d48f2d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.398575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.419092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c9fe25-69fa-45e1-967c-6cffd1f28bd0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.423859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.424932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.425224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.425515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] PATH_INFO: `/v3/projects/8899c92c0c9b4b69af1d9aa2d0d8ee03` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.426841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8899c92c0c9b4b69af1d9aa2d0d8ee03)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.428802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.448534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f435937946eb47c0bab7e0e6b5d48f2d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.455630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f435937946eb47c0bab7e0e6b5d48f2d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.456489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f435937946eb47c0bab7e0e6b5d48f2d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.457531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f435937946eb47c0bab7e0e6b5d48f2d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.470792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-afd5116d-4400-43c9-bd59-726cdee1ecd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f435937946eb47c0bab7e0e6b5d48f2d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.479308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8899c92c0c9b4b69af1d9aa2d0d8ee03', 'request_id': 'req-0308b774-c435-427f-93ea-42f14487c51c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.484378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1036/2094] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:45 2022] DELETE /identity/v3/users/f435937946eb47c0bab7e0e6b5d48f2d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:45.485416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8899c92c0c9b4b69af1d9aa2d0d8ee03', 'request_id': 'req-0308b774-c435-427f-93ea-42f14487c51c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.488875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.489167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.489474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.491657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8899c92c0c9b4b69af1d9aa2d0d8ee03', 'request_id': 'req-0308b774-c435-427f-93ea-42f14487c51c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.504494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] The token cache is being invalidate because project 8899c92c0c9b4b69af1d9aa2d0d8ee03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:45.504785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.505397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0308b774-c435-427f-93ea-42f14487c51c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:45.505843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:45.506966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1059/2095] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:45 2022] DELETE /identity/v3/projects/8899c92c0c9b4b69af1d9aa2d0d8ee03 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:45.510415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.511823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af29109-d0a2-483e-ae48-3e82198654d3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.512127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af29109-d0a2-483e-ae48-3e82198654d3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.512406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af29109-d0a2-483e-ae48-3e82198654d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.530617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0af29109-d0a2-483e-ae48-3e82198654d3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:45.531848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0af29109-d0a2-483e-ae48-3e82198654d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.537198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:45.568487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1037/2096] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:45.577981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.579407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.592782 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1060/2097] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:45.600651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.600940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.601216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.621056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:45.622275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:45.632053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1061/2098] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:45.637771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.646539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:45.654617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fc6688-4563-4e11-9c51-4dcb0a4e1e29 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.655207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5046608f-5956-4066-b8ce-37f3d7fd5c06 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.661213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.662301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.663964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.663964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.666619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.667410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.667698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.668141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.669114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:45.669457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.671700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.682171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.683798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.687981 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1062/2099] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:02:45.692070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.713136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1038/2100] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:02:45.716419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc4a61a-0f9c-4046-9bfd-ae657cb633d5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.727840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.728730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.729119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.729498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.730292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.733639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.765983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1063/2101] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:45 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:45.769982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.794949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d465f0d5-5740-4621-8c99-3542955cc993 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.909984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.911123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.911281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.911507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.912348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.915286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.953294 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1039/2102] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:45 2022] POST /identity/v3/users => generated 282 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:45.953733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:45.965029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8856b989-5ca2-4ccb-bc1d-30513c70bd5f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:45.970713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:45.971704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:45.971987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:45.972338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:45.973070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:45.974674 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:45.986541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1064/2103] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:45.990613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:46.010242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcad86-32fa-4e5a-a045-2105b702f56c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:46.022213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096f74b0-88a9-4b03-ad88-fd12dfe0ca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:46.023388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f74b0-88a9-4b03-ad88-fd12dfe0ca1f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:46.025807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f74b0-88a9-4b03-ad88-fd12dfe0ca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:46.026256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096f74b0-88a9-4b03-ad88-fd12dfe0ca1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:46.027192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f74b0-88a9-4b03-ad88-fd12dfe0ca1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:46.030714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096f74b0-88a9-4b03-ad88-fd12dfe0ca1f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:46.066106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1040/2104] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:45 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:46.067682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:46.084276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eefb1-2233-4156-a5cc-b2839264f874 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:46.096639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:46.098670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:46.100329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:46.115491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] PATH_INFO: `/v3/projects/190f21b1b77e4186ac4f84ab5b788ebf/users/c955c77aa06d4663b815b9cfa62c7534/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:46.167579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=190f21b1b77e4186ac4f84ab5b788ebf, user_id=c955c77aa06d4663b815b9cfa62c7534, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:46.171964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:46.193432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1065/2105] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:46 2022] PUT /identity/v3/projects/190f21b1b77e4186ac4f84ab5b788ebf/users/c955c77aa06d4663b815b9cfa62c7534/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:46.203610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:46.203989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:46.204364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:46.253454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] MFA Rules not processed for user `c955c77aa06d4663b815b9cfa62c7534`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:46.254782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:46.364365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:46.366191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:46.374147 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1041/2106] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:46 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:46.386545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:46.388444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:46.466359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eb54a9-8b97-4dda-b657-d0a49b5ceae9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:46.470557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744f209-d1f8-4ce7-b404-ce019f343bf7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:46.470690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:46.471330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:46.471632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:46.472729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:46.477317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:46.478268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:46.478582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:46.479046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] PATH_INFO: `/v3/projects/a396fdf121ea4308a2efe49416bd1c96` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:46.480915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a396fdf121ea4308a2efe49416bd1c96)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:46.481639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:46.482320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:46.483701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:46.494183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1042/2107] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:46 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:46.525562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:46.542622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a396fdf121ea4308a2efe49416bd1c96', 'request_id': 'req-3fe58459-08a1-4b19-abb4-669afdd58296'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.552416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a396fdf121ea4308a2efe49416bd1c96', 'request_id': 'req-3fe58459-08a1-4b19-abb4-669afdd58296'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.561101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a396fdf121ea4308a2efe49416bd1c96', 'request_id': 'req-3fe58459-08a1-4b19-abb4-669afdd58296'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.575923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] The token cache is being invalidate because project a396fdf121ea4308a2efe49416bd1c96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:46.576224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.576889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.581717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1066/2108] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:46 2022] DELETE /identity/v3/projects/a396fdf121ea4308a2efe49416bd1c96 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:46.582932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:46.589187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:46.596013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb07e5e-f2b7-48f1-8651-95ebb3913c35 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:46.666608 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:46.667533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:46.667927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:46.668094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] PATH_INFO: `/v3/users/f72b99b42f1b44bd826de2e84535bbaf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:46.669830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72b99b42f1b44bd826de2e84535bbaf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:46.671475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:46.725180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72b99b42f1b44bd826de2e84535bbaf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.732872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72b99b42f1b44bd826de2e84535bbaf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.733663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72b99b42f1b44bd826de2e84535bbaf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.735571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72b99b42f1b44bd826de2e84535bbaf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.751288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72b99b42f1b44bd826de2e84535bbaf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:46.766769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1043/2109] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:46 2022] DELETE /identity/v3/users/f72b99b42f1b44bd826de2e84535bbaf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:02:47.112591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:47.114606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:47.130225 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe58459-08a1-4b19-abb4-669afdd58296 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:47.189725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:47.190502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:47.190788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:47.191058 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] PATH_INFO: `/v3/projects/344574c939c8406cbdd1c6debe7a9668` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:47.192249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=344574c939c8406cbdd1c6debe7a9668)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:47.194155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:47.239399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344574c939c8406cbdd1c6debe7a9668', 'request_id': 'req-b4491260-53f6-4584-876a-26cfcf0c19ba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.245262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344574c939c8406cbdd1c6debe7a9668', 'request_id': 'req-b4491260-53f6-4584-876a-26cfcf0c19ba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.251194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344574c939c8406cbdd1c6debe7a9668', 'request_id': 'req-b4491260-53f6-4584-876a-26cfcf0c19ba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.264186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] The token cache is being invalidate because project 344574c939c8406cbdd1c6debe7a9668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:47.264495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.265154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.266672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1067/2110] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:47 2022] DELETE /identity/v3/projects/344574c939c8406cbdd1c6debe7a9668 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:47.270817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:47.272360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:47.285192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7cf62-f0fe-49ce-8c1a-06eaa9d19392 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:47.345155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:47.346041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:47.346325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:47.346601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] PATH_INFO: `/v3/users/366121896c82453a8f74796c9f37ea25` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:47.347884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=366121896c82453a8f74796c9f37ea25)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:47.349809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:47.376688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:47.413674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '366121896c82453a8f74796c9f37ea25'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.430973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '366121896c82453a8f74796c9f37ea25'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.431752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '366121896c82453a8f74796c9f37ea25'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.432731 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '366121896c82453a8f74796c9f37ea25'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.446684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-13f27307-c53f-4980-9aa7-742e82ff6cc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '366121896c82453a8f74796c9f37ea25'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.465483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1044/2111] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:47 2022] DELETE /identity/v3/users/366121896c82453a8f74796c9f37ea25 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:47.725415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:47.727416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:47.740525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4491260-53f6-4584-876a-26cfcf0c19ba admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:47.790870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:47.798973 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:47.799883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:47.800373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:47.800961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] PATH_INFO: `/v3/projects/ad8abf874f8c42ee99c77d6962252ab1` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:47.802969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad8abf874f8c42ee99c77d6962252ab1)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:47.805166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:47.851768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad8abf874f8c42ee99c77d6962252ab1', 'request_id': 'req-eab1bb8f-fa0b-4458-acdd-e2625b85857c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.857263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad8abf874f8c42ee99c77d6962252ab1', 'request_id': 'req-eab1bb8f-fa0b-4458-acdd-e2625b85857c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.863877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad8abf874f8c42ee99c77d6962252ab1', 'request_id': 'req-eab1bb8f-fa0b-4458-acdd-e2625b85857c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.876736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] The token cache is being invalidate because project ad8abf874f8c42ee99c77d6962252ab1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:47.877028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.877657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-eab1bb8f-fa0b-4458-acdd-e2625b85857c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:47.879200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1068/2112] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:47 2022] DELETE /identity/v3/projects/ad8abf874f8c42ee99c77d6962252ab1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:47.885179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:47.885472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:47.885748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:47.909143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:47.910430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:47.972325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1045/2113] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:47.980490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:47.981019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:47.981302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.001023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:48.002233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:48.010946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1069/2114] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:48.015189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.026601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa385b-11c1-4979-9ef0-04b9c1f62506 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.031308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.031977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.032265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.032546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.033313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:02:48.033624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.035225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.047088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1046/2115] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:02:48.051364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.060479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d33935-f428-48b5-b1e6-cd15a414c6a4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.065434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.066079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.066366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.066643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.067360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.069249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.094831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1070/2116] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:48 2022] POST /identity/v3/projects => generated 407 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:02:48.098865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.108801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1adf92e-b47d-4a6b-8a0f-afd56fc5efa2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.163693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.164395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.164680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.164953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.165632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.167544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.189051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1047/2117] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:48 2022] POST /identity/v3/users => generated 304 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:48.190196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.200695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def73045-8ce5-4901-8f5d-6428c43ad661 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.205496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.206154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.206434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.206709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.207475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.209065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.220007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1071/2118] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:48.223774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.233507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aa8db-19c0-405a-a935-5950f8836bb3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.238098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159d657-2813-436e-aaf7-a32f415b2504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.238750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159d657-2813-436e-aaf7-a32f415b2504 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.239034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159d657-2813-436e-aaf7-a32f415b2504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.239357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159d657-2813-436e-aaf7-a32f415b2504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.240082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159d657-2813-436e-aaf7-a32f415b2504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.241659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159d657-2813-436e-aaf7-a32f415b2504 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.251499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1048/2119] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:48.255494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.265809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c702d344-4401-4d51-8674-db9863dda647 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.270690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.271614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.271903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.272185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] PATH_INFO: `/v3/projects/3b95f4ed51754df88e1459916579baec/users/3d045c2b995b49a0ac2fed62f2f13980/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.286734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b95f4ed51754df88e1459916579baec, user_id=3d045c2b995b49a0ac2fed62f2f13980, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.289946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.302667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1072/2120] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:48 2022] PUT /identity/v3/projects/3b95f4ed51754df88e1459916579baec/users/3d045c2b995b49a0ac2fed62f2f13980/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:48.314198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.314489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.314769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.328470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:48.345006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] MFA Rules not processed for user `3d045c2b995b49a0ac2fed62f2f13980`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:48.346221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:48.408504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1049/2121] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:48 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:48.412989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.474834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86170ebd-d6a8-496d-81a4-eba8288100c8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.480149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.480817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.481103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.481382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.482074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.484022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.520531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1073/2122] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:02:48 2022] POST /identity/v3/projects => generated 407 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:48.521847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.534000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6f19f7-2b38-4c54-b3c0-9facabdf45de None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.602963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.603775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.604206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.604405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.605106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.607028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.633991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1050/2123] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:02:48 2022] POST /identity/v3/users => generated 310 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:02:48.637869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.648560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cd6e32-48d7-4b31-aa8f-8fe7c2e8057d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.666448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.667137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.667460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.667828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.668738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.670393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.671928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:48.685463 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1074/2124] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:02:48.689482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.700670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d170a-2d05-479e-b707-b0f1c0d64139 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.705191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.706064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.706348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.706622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] PATH_INFO: `/v3/projects/288f884ce7d84a80b4b789d5426fdb09/users/8a66b20d3ca842d79462d53f76abfd38/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.721192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288f884ce7d84a80b4b789d5426fdb09, user_id=8a66b20d3ca842d79462d53f76abfd38, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.723877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.737050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1051/2125] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:48 2022] PUT /identity/v3/projects/288f884ce7d84a80b4b789d5426fdb09/users/8a66b20d3ca842d79462d53f76abfd38/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:48.741021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.751459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afb9c5-5db7-4ccd-8f79-8ab5dcc984fc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.807818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a3ae9d-2a6a-4060-8480-a82d39ed833c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.808472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ae9d-2a6a-4060-8480-a82d39ed833c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.808754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ae9d-2a6a-4060-8480-a82d39ed833c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.809027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a3ae9d-2a6a-4060-8480-a82d39ed833c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.809740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3ae9d-2a6a-4060-8480-a82d39ed833c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.811422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a3ae9d-2a6a-4060-8480-a82d39ed833c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.826895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1075/2126] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:02:48.828515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.838830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35622be-ab50-4f17-8c87-e8b773d90b71 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:48.843656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:48.844532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.844822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.845094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] PATH_INFO: `/v3/projects/288f884ce7d84a80b4b789d5426fdb09/users/8a66b20d3ca842d79462d53f76abfd38/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.847037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] RBAC: Authorizing `identity:create_grant(project_id=288f884ce7d84a80b4b789d5426fdb09, user_id=8a66b20d3ca842d79462d53f76abfd38, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:48.850261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:48.861080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1052/2127] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:02:48 2022] PUT /identity/v3/projects/288f884ce7d84a80b4b789d5426fdb09/users/8a66b20d3ca842d79462d53f76abfd38/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:48.872479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:48.872777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:48.873064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:48.898289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] MFA Rules not processed for user `8a66b20d3ca842d79462d53f76abfd38`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:48.899525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:48.969850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1076/2128] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:48 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:48.980604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:48.982345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:49.035534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:49.039296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ff5c3-59db-40af-9292-4191f6b39412 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:49.043778 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:49.044422 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:49.044711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:49.044988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:49.054071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:49.055522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:49.064978 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1053/2129] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:48 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:49.214013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:49.223664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fc698c-c948-4812-ac91-bdb20fd01b3f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:49.227239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc329d5-98e0-4465-ac93-ce0bf2513527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:49.227900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc329d5-98e0-4465-ac93-ce0bf2513527 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:49.228186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc329d5-98e0-4465-ac93-ce0bf2513527 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:49.228464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc329d5-98e0-4465-ac93-ce0bf2513527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:49.283581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc329d5-98e0-4465-ac93-ce0bf2513527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:49.285018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc329d5-98e0-4465-ac93-ce0bf2513527 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:49.293833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1077/2130] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:49 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:58.960431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:59.030734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2763fa-cc09-484d-bba9-2f68e3ce8939 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:59.035897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:59.036795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:59.037078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:59.037351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] PATH_INFO: `/v3/users/3d045c2b995b49a0ac2fed62f2f13980` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:59.038558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d045c2b995b49a0ac2fed62f2f13980)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:59.041109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:59.088502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d045c2b995b49a0ac2fed62f2f13980'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.095885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d045c2b995b49a0ac2fed62f2f13980'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.096625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d045c2b995b49a0ac2fed62f2f13980'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.097219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d045c2b995b49a0ac2fed62f2f13980'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.114865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d045c2b995b49a0ac2fed62f2f13980'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.129186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1054/2131] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:58 2022] DELETE /identity/v3/users/3d045c2b995b49a0ac2fed62f2f13980 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:59.133849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:59.134288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:59.134405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:59.172237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:59.174294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:59.185061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:59.237108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1078/2132] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:02:59.249464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:59.250936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:59.340739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18ce48-935a-4f24-8295-cd62f32b0fe0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:59.347022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236c4ce-94c8-4917-96a8-e466c67ce952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:59.347884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c4ce-94c8-4917-96a8-e466c67ce952 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:59.348258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c4ce-94c8-4917-96a8-e466c67ce952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:59.348682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236c4ce-94c8-4917-96a8-e466c67ce952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:59.360269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c4ce-94c8-4917-96a8-e466c67ce952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:59.361692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236c4ce-94c8-4917-96a8-e466c67ce952 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:59.371308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1055/2133] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:02:59.686879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:59.688481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:59.704004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b884e89-4b61-4af5-bcbb-74923cf34c24 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:59.708680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:59.709552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:59.710016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:59.710378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] PATH_INFO: `/v3/projects/3b95f4ed51754df88e1459916579baec` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:59.711859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b95f4ed51754df88e1459916579baec)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:59.714464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:59.754037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:59.754342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:59.754629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:59.767891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b95f4ed51754df88e1459916579baec', 'request_id': 'req-b32bb0e4-f25e-45d5-a1c5-066f7181b293'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.774183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:02:59.775532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:59.775819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b95f4ed51754df88e1459916579baec', 'request_id': 'req-b32bb0e4-f25e-45d5-a1c5-066f7181b293'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.783229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b95f4ed51754df88e1459916579baec', 'request_id': 'req-b32bb0e4-f25e-45d5-a1c5-066f7181b293'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.786468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:02:59.813645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] The token cache is being invalidate because project 3b95f4ed51754df88e1459916579baec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:02:59.813947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.816151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:02:59.818781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1079/2134] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:02:59 2022] DELETE /identity/v3/projects/3b95f4ed51754df88e1459916579baec => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:02:59.823672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:59.830235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:02:59.889996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1056/2135] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:02:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:02:59.933145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32bb0e4-f25e-45d5-a1c5-066f7181b293 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:02:59.941725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:02:59.942543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:02:59.942825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:02:59.943116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] PATH_INFO: `/v3/users/8a66b20d3ca842d79462d53f76abfd38` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:02:59.944604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a66b20d3ca842d79462d53f76abfd38)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:02:59.946886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:02:59.947789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:02:59.949362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.041094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:00.047406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a66b20d3ca842d79462d53f76abfd38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.051956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fe754-ae87-4888-9db8-07bb5e1cf9e6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:00.059867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a66b20d3ca842d79462d53f76abfd38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.061210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a66b20d3ca842d79462d53f76abfd38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.062165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:00.062707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a66b20d3ca842d79462d53f76abfd38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.064036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.064507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.064853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.067049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.079099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:00.079687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6b3923a0-bdf4-45df-a686-db587ef9b9ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a66b20d3ca842d79462d53f76abfd38'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.080979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:00.090309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1057/2136] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:02:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 06:03:00.098576 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1080/2137] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:02:59 2022] DELETE /identity/v3/users/8a66b20d3ca842d79462d53f76abfd38 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:00.212280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:00.214206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.249189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d16638c-8980-4f29-818b-13bda71083c8 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:00.315979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:00.317006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.318051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.318511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] PATH_INFO: `/v3/users/c955c77aa06d4663b815b9cfa62c7534` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.320340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c955c77aa06d4663b815b9cfa62c7534)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:00.322839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:00.409649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c955c77aa06d4663b815b9cfa62c7534'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.414183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:00.428789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c955c77aa06d4663b815b9cfa62c7534'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.430526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c955c77aa06d4663b815b9cfa62c7534'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.431563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c955c77aa06d4663b815b9cfa62c7534'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.451735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c955c77aa06d4663b815b9cfa62c7534'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.479756 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1058/2138] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:00 2022] DELETE /identity/v3/users/c955c77aa06d4663b815b9cfa62c7534 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:00.484320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.484846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.485105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.519041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:00.520505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.573009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:00.576297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.657434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1081/2139] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:00.669051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:00.670874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.702843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06317162-81d3-48dd-81b7-f6ee9c3fe2b2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:00.711023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:00.711915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.712199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.712475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] PATH_INFO: `/v3/projects/288f884ce7d84a80b4b789d5426fdb09` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.714046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=288f884ce7d84a80b4b789d5426fdb09)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:00.716931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:00.746655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cdc6a0-916c-4555-ac6a-6412c2b6bbaa None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:00.750806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b0d74a-c353-428c-9925-bb78d6aca42f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:00.751526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0d74a-c353-428c-9925-bb78d6aca42f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.751810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0d74a-c353-428c-9925-bb78d6aca42f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.752125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b0d74a-c353-428c-9925-bb78d6aca42f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.763922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0d74a-c353-428c-9925-bb78d6aca42f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:00.765347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b0d74a-c353-428c-9925-bb78d6aca42f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:00.780777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1082/2140] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:00.781482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '288f884ce7d84a80b4b789d5426fdb09', 'request_id': 'req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.790191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '288f884ce7d84a80b4b789d5426fdb09', 'request_id': 'req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.796929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '288f884ce7d84a80b4b789d5426fdb09', 'request_id': 'req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.812161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f8c5-4535-44a0-8158-9ad887bf30bd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.812445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f8c5-4535-44a0-8158-9ad887bf30bd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.812728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58f8c5-4535-44a0-8158-9ad887bf30bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.813759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] The token cache is being invalidate because project 288f884ce7d84a80b4b789d5426fdb09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:00.813759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.814474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-10c3c4ab-f283-49f2-8d28-e1f1310e16d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:00.822309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1059/2141] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:00 2022] DELETE /identity/v3/projects/288f884ce7d84a80b4b789d5426fdb09 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:03:00.822309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.822309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.822309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.839739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9a58f8c5-4535-44a0-8158-9ad887bf30bd None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:00.841419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9a58f8c5-4535-44a0-8158-9ad887bf30bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.845893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:00.847352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.916228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:00.926520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1083/2142] 10.222.0.45 () {58 vars in 967 bytes} [Mon Jun 20 06:03:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:00.928052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1060/2143] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:00.936414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:00.936797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:00.937203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:00.948203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:00.949660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.956090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:00.971515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:00.973311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:00.983405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1084/2144] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:00.987606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:00.997494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f571d9-685e-44fd-808d-485b2f5dddc5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.002563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.003336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.003881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.004032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.004839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:01.005226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.007227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.029056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1085/2145] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:01.030152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.042549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092220ed-4bb2-4473-b1b5-d71642586f1d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.047772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.048522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.048886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.049240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.050016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.052041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.052541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df71c8-2d33-47e4-83cc-74d90d5b95dd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.059662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.060673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.060969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.061251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.073554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.075391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.087875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1086/2146] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:01.091483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1061/2147] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:03:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:03:01.101824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.112504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.114593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.156564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42241f-c838-4075-a124-4450e944e3f0 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.196623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df83cd-c194-4e1c-a6da-8810e9963f91 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.202495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.203884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.204199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.204635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.205373 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.207637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.237129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1087/2148] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:01.241579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.262338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114dba57-5b73-47ce-9eeb-857b2547e66d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.264286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.265315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.265614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.265966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] PATH_INFO: `/v3/projects/190f21b1b77e4186ac4f84ab5b788ebf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.267400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] RBAC: Authorizing `identity:delete_project(project_id=190f21b1b77e4186ac4f84ab5b788ebf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.268961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.269626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.270022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.270454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.271010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.271779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.273920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.286375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1088/2149] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:03:01.290914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.303026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e4c09-6a16-4b22-bac3-7d5dce5b5c04 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.309584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.310557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.310690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.311116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.311697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.313279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.332063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1089/2150] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:03:01.335950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '190f21b1b77e4186ac4f84ab5b788ebf', 'request_id': 'req-7e0b707f-d1b5-4f67-a655-7c3ccc145319'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:01.340286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.342879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '190f21b1b77e4186ac4f84ab5b788ebf', 'request_id': 'req-7e0b707f-d1b5-4f67-a655-7c3ccc145319'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:01.352152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a395a5d-2a84-4895-824c-4c415cfbae2f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.352377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '190f21b1b77e4186ac4f84ab5b788ebf', 'request_id': 'req-7e0b707f-d1b5-4f67-a655-7c3ccc145319'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:01.366163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] The token cache is being invalidate because project 190f21b1b77e4186ac4f84ab5b788ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:01.366711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:01.367141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-7e0b707f-d1b5-4f67-a655-7c3ccc145319 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:01.369375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1062/2151] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:01 2022] DELETE /identity/v3/projects/190f21b1b77e4186ac4f84ab5b788ebf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:03:01.374028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c746-0c84-487e-ae9c-b24567413370 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.374323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c746-0c84-487e-ae9c-b24567413370 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.374604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4197c746-0c84-487e-ae9c-b24567413370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.394122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4197c746-0c84-487e-ae9c-b24567413370 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:01.395648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4197c746-0c84-487e-ae9c-b24567413370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.408873 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:01.414494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200b16df-a422-43a9-a0c5-0f8cb4b3726e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.415526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b16df-a422-43a9-a0c5-0f8cb4b3726e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.415865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b16df-a422-43a9-a0c5-0f8cb4b3726e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.416170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200b16df-a422-43a9-a0c5-0f8cb4b3726e admin admin] PATH_INFO: `/v3/projects/7cecdd3d4d754134b651cb151bb86f53/users/e8e3da322b814874b1843e23dbc73cd6/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.431503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b16df-a422-43a9-a0c5-0f8cb4b3726e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cecdd3d4d754134b651cb151bb86f53, user_id=e8e3da322b814874b1843e23dbc73cd6, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.434214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200b16df-a422-43a9-a0c5-0f8cb4b3726e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.447846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1090/2152] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:01 2022] PUT /identity/v3/projects/7cecdd3d4d754134b651cb151bb86f53/users/e8e3da322b814874b1843e23dbc73cd6/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:01.449103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926df4-e9a9-4d75-b9c6-91df218a0e82 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.449397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926df4-e9a9-4d75-b9c6-91df218a0e82 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.449679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef926df4-e9a9-4d75-b9c6-91df218a0e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.470544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ef926df4-e9a9-4d75-b9c6-91df218a0e82 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:01.471799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ef926df4-e9a9-4d75-b9c6-91df218a0e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.479229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1063/2153] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:01.482363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:01.484874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.485322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.485770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.530912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] MFA Rules not processed for user `e8e3da322b814874b1843e23dbc73cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:01.532366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.532584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1091/2154] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:01.533745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.534077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.534292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.552447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:01.553660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.603299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1064/2155] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:01.603995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.605290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.625118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1092/2156] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:01.625118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.625118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:01.681222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b766af-f530-491f-b5de-e97eeae94c36 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.682845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf0850-6dfc-4afa-a408-4fc5078c0156 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.686313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.686664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.686965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.687269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.687364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.687561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.687688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.687892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.698013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.698388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.699718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.700033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.710221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1093/2158] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:01.711831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.717990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1065/2157] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:03:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:03:01.729256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433f4fd-3823-48af-8653-9da16f1e8453 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.733890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.734896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.735745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.735745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.736207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:01.736521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.738142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.751613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1094/2159] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:01.755960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.770473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03053563-50f2-4a74-b2cc-66eb855ae614 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.798890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.800009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.800290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.800572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.801301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.803419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.859274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1066/2160] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:01.862354 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.875012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc33aed-a20e-4e47-9ee4-affb9016a3aa admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:01.932375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:01.943554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:01.944335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:01.944691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:01.945146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:01.945876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:01.948797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:01.975537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1095/2161] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:01 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:01.977775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:01.990192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be47e38-ec0d-46a5-8928-7cc637d6a813 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.000311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.001180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.001635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.001986 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.002795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.004536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.018612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1067/2162] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:02.023667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.036612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202d837-416a-489c-b345-cd2eebd3f85b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.044078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c73970-0fbe-4cc6-9a3d-64f037f1f084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.044714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c73970-0fbe-4cc6-9a3d-64f037f1f084 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.044997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c73970-0fbe-4cc6-9a3d-64f037f1f084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.045274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c73970-0fbe-4cc6-9a3d-64f037f1f084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.046006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c73970-0fbe-4cc6-9a3d-64f037f1f084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.047661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c73970-0fbe-4cc6-9a3d-64f037f1f084 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.060727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1096/2163] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:03:02.064791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.073859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:02.082309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2971200e-2db0-459a-a1b2-21b1c3f2f312 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.095863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc3a62-12b8-4aea-9e93-21c1e31ffea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.097567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc3a62-12b8-4aea-9e93-21c1e31ffea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.098007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc3a62-12b8-4aea-9e93-21c1e31ffea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.098433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc3a62-12b8-4aea-9e93-21c1e31ffea1 admin admin] PATH_INFO: `/v3/projects/8bdd02d892a24a40b4e17a4063a08765/users/455a5bf6e4e34afcaebdde3beaa619ea/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.123941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc3a62-12b8-4aea-9e93-21c1e31ffea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bdd02d892a24a40b4e17a4063a08765, user_id=455a5bf6e4e34afcaebdde3beaa619ea, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.126795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc3a62-12b8-4aea-9e93-21c1e31ffea1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.153938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1068/2164] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:02 2022] PUT /identity/v3/projects/8bdd02d892a24a40b4e17a4063a08765/users/455a5bf6e4e34afcaebdde3beaa619ea/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:02.165698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.166017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.166265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.193035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.193330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.193614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.200060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] MFA Rules not processed for user `455a5bf6e4e34afcaebdde3beaa619ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:02.201384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:02.215068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:02.216588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:02.276559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1097/2165] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:02.281235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.288350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1069/2166] 10.222.0.45 () {58 vars in 967 bytes} [Mon Jun 20 06:03:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:02.302284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.352795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ca4008-4fc3-4fe0-b637-b6ce5bfe9d1c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.357677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.358334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.358625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.358905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.359650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.361577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.389705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4fac8c-ab76-48a8-bb73-87520eaeaa6d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.390497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1098/2167] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:02 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:03:02.394370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.450426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:02.468185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.469038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.469421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.470098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.473892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75476d58-e284-4e5e-8c04-e015932240b8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.479294 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.480066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.480349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.480668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.481586 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.483628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.501235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:02.503584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1099/2168] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:02 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:03:02.508030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.517635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2917dff-b362-416a-a071-7e6f443ee1cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.523662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.524331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.524626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.524916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.525669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.527244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.531998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.533574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.538759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1100/2169] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:03:02.542159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1070/2170] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:03:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 20 06:03:02.543186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.554369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ead6-3785-4362-bef0-4eeea159d147 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.561085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.562446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.562778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.563101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] PATH_INFO: `/v3/projects/031d6f9897da43de9bc42bd52abc93fc/users/b5d7cb9dab374be0bfc9838a1042ebba/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.579890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d6f9897da43de9bc42bd52abc93fc, user_id=b5d7cb9dab374be0bfc9838a1042ebba, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.582597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.596878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1101/2171] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:02 2022] PUT /identity/v3/projects/031d6f9897da43de9bc42bd52abc93fc/users/b5d7cb9dab374be0bfc9838a1042ebba/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:02.601083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.612178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f97ea3e-d9d2-47c7-a60b-65940e90d296 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.677649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa77601-3e12-4f0d-a6bd-8cc73e936604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.678407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa77601-3e12-4f0d-a6bd-8cc73e936604 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.678737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa77601-3e12-4f0d-a6bd-8cc73e936604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.679017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa77601-3e12-4f0d-a6bd-8cc73e936604 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.680354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa77601-3e12-4f0d-a6bd-8cc73e936604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.682324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa77601-3e12-4f0d-a6bd-8cc73e936604 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.698611 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1071/2172] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:02.699842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.710501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a1eb84-ee8d-422b-98d3-50089ec06bb9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.716237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.716915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.717201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.717477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] PATH_INFO: `/v3/projects/031d6f9897da43de9bc42bd52abc93fc/users/b5d7cb9dab374be0bfc9838a1042ebba/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.719516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] RBAC: Authorizing `identity:create_grant(project_id=031d6f9897da43de9bc42bd52abc93fc, user_id=b5d7cb9dab374be0bfc9838a1042ebba, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.722164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.733669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1102/2173] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:02 2022] PUT /identity/v3/projects/031d6f9897da43de9bc42bd52abc93fc/users/b5d7cb9dab374be0bfc9838a1042ebba/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:02.745795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.746089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.746371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.775028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] MFA Rules not processed for user `b5d7cb9dab374be0bfc9838a1042ebba`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:02.776268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:02.878267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1072/2174] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:02 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:02.886972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:02.948261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48feac56-89ed-4d50-8948-17a6cfa602da admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:02.951961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:02.952637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:02.952950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:02.953226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:02.963690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:02.965147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:02.975070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1103/2175] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:03.064283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:03.074430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36d2c0d-7a10-4aac-b73e-33e5815deef1 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:03.078047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57c350c-1a30-4a4f-83ad-6606ae621aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:03.078688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c350c-1a30-4a4f-83ad-6606ae621aff service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:03.078969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c350c-1a30-4a4f-83ad-6606ae621aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:03.079265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c350c-1a30-4a4f-83ad-6606ae621aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:03.130026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:03.142336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c350c-1a30-4a4f-83ad-6606ae621aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:03.143957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57c350c-1a30-4a4f-83ad-6606ae621aff service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:03.155418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1073/2176] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:03:07.238176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:07.239762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:07.306610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a97c6ea-6713-4d9b-94ec-2f86ce03556b service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:07.311529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:07.312427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:07.312816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:07.313088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] PATH_INFO: `/v3/users/1e11d00ff8724073a0c9b229cb44167f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:07.314237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e11d00ff8724073a0c9b229cb44167f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:07.316206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:07.349523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:07.368316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e11d00ff8724073a0c9b229cb44167f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:07.373253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e11d00ff8724073a0c9b229cb44167f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:07.373991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e11d00ff8724073a0c9b229cb44167f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:07.374617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e11d00ff8724073a0c9b229cb44167f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:07.387044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e11d00ff8724073a0c9b229cb44167f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:07.399671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1104/2177] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:07 2022] DELETE /identity/v3/users/1e11d00ff8724073a0c9b229cb44167f => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:07.404112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:07.404612 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:07.404711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:07.421146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:07.422373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:07.482690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1074/2178] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:07.492935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:07.494334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:07.554360 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df12e25-af06-4506-9646-0b6d04706ba9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:07.558047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f377b0-cf65-47e1-879d-8bc36d1d0a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:07.558698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f377b0-cf65-47e1-879d-8bc36d1d0a16 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:07.558985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f377b0-cf65-47e1-879d-8bc36d1d0a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:07.559285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f377b0-cf65-47e1-879d-8bc36d1d0a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:07.568596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f377b0-cf65-47e1-879d-8bc36d1d0a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:07.570602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f377b0-cf65-47e1-879d-8bc36d1d0a16 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:07.580563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1105/2179] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:07.804952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:07.807295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:07.836191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02b179-4904-405a-9515-b93b6360fd63 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:07.842832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:07.844701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:07.845337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:07.845831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] PATH_INFO: `/v3/projects/7ebe4d344e324bb68dec94a296d31b5a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:07.848184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ebe4d344e324bb68dec94a296d31b5a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:07.850713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:07.988326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ebe4d344e324bb68dec94a296d31b5a', 'request_id': 'req-5fca624c-ad80-45ca-ae21-59c8abbd013d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:07.998131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ebe4d344e324bb68dec94a296d31b5a', 'request_id': 'req-5fca624c-ad80-45ca-ae21-59c8abbd013d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:08.009329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ebe4d344e324bb68dec94a296d31b5a', 'request_id': 'req-5fca624c-ad80-45ca-ae21-59c8abbd013d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:08.026674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] The token cache is being invalidate because project 7ebe4d344e324bb68dec94a296d31b5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:08.026971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:08.027666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5fca624c-ad80-45ca-ae21-59c8abbd013d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:08.031150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1075/2180] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:07 2022] DELETE /identity/v3/projects/7ebe4d344e324bb68dec94a296d31b5a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:08.036328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.037209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.037209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.067754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:08.070633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:08.177435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1106/2181] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:08.187118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.187809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.187809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.212079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:08.213541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:08.225198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1076/2182] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:08.229183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.241290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4966c4-cfff-4161-b75c-5df5b6256878 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.249747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.250847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.251206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.251866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.252828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:08.253851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.255313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.272180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1107/2183] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:08.276787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.290044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a66238-0025-4462-a5ef-9f2e80591a2b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.304362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.305096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.305385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.305854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.306421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.308848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.342691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1077/2184] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:08 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:08.347088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.357568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0597c7e2-c542-480d-ad06-fd35bcc11a49 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.408469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:08.435923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.436881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.437243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.437689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.438489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.441252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.462819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1108/2185] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:08 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:08.467398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.478850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43e8b8a-8cb7-43f7-a1e7-8d54d94655db admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.484235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.485222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.485515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.485794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.489715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.489715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.504622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1078/2186] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:08.508673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.520020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d53b5-2639-4df5-a239-084ab7065029 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.525666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ae1268-19a3-4d46-83c7-d7db411cc674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.526380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae1268-19a3-4d46-83c7-d7db411cc674 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.527970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae1268-19a3-4d46-83c7-d7db411cc674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.527970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ae1268-19a3-4d46-83c7-d7db411cc674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.527970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae1268-19a3-4d46-83c7-d7db411cc674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.529542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ae1268-19a3-4d46-83c7-d7db411cc674 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.542041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1109/2187] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:08.546095 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.560594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a07229-9784-48c3-8330-1d2c9b3a2235 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.566640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.567729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.568044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.568282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] PATH_INFO: `/v3/projects/75f7c2d4c0e742cd93e51fee7df062f0/users/6490c3d52b5a45aea9236e171ae94770/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.586297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f7c2d4c0e742cd93e51fee7df062f0, user_id=6490c3d52b5a45aea9236e171ae94770, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.589171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.603366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1079/2188] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:08 2022] PUT /identity/v3/projects/75f7c2d4c0e742cd93e51fee7df062f0/users/6490c3d52b5a45aea9236e171ae94770/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:03:08.608920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076d9fe-bd7b-48d2-bfcd-069d92e11d98 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.609216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076d9fe-bd7b-48d2-bfcd-069d92e11d98 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.609500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076d9fe-bd7b-48d2-bfcd-069d92e11d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.628533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d076d9fe-bd7b-48d2-bfcd-069d92e11d98 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:08.629815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d076d9fe-bd7b-48d2-bfcd-069d92e11d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:08.696045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1110/2189] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:08.705454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:08.707409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:08.719055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:08.771098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d1ea83-a8f9-4884-9289-5a02c2a54eb3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:08.775286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:08.775940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:08.776225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:08.776524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:08.785473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:08.786878 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:08.798641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1080/2190] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:09.541573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242258d-c84f-4d23-b094-33f04ba1a3cc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:09.542065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242258d-c84f-4d23-b094-33f04ba1a3cc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:09.542184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242258d-c84f-4d23-b094-33f04ba1a3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:09.578037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1242258d-c84f-4d23-b094-33f04ba1a3cc None None] MFA Rules not processed for user `6490c3d52b5a45aea9236e171ae94770`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:09.579185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1242258d-c84f-4d23-b094-33f04ba1a3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:09.687680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1111/2191] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:09.688844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:09.690795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:09.758661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182008d0-217d-4d6d-83de-ccdeeedc4706 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:09.769915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:09.770961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:09.771441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:09.771917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:09.784112 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:09.785581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:09.795567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1081/2192] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:03:13.271748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817c041-fac8-408c-ba7e-43b21c55c113 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:13.272263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817c041-fac8-408c-ba7e-43b21c55c113 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:13.272583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817c041-fac8-408c-ba7e-43b21c55c113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:13.291382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a817c041-fac8-408c-ba7e-43b21c55c113 None None] MFA Rules not processed for user `6490c3d52b5a45aea9236e171ae94770`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:13.293598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a817c041-fac8-408c-ba7e-43b21c55c113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:13.300798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1112/2193] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:13.956438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:13.957939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:13.971912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9377e1c-786d-4066-ab67-f106aa0fdae4 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:13.976860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:13.977701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:13.977995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:13.978284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] PATH_INFO: `/v3/users/e8e3da322b814874b1843e23dbc73cd6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:13.979514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e3da322b814874b1843e23dbc73cd6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:13.981443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:14.030561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e3da322b814874b1843e23dbc73cd6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.037825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e3da322b814874b1843e23dbc73cd6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.038533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e3da322b814874b1843e23dbc73cd6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.039139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e3da322b814874b1843e23dbc73cd6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.052395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e3da322b814874b1843e23dbc73cd6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.066489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1082/2194] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:13 2022] DELETE /identity/v3/users/e8e3da322b814874b1843e23dbc73cd6 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:14.071771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.072167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.072456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.089656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:14.091142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:14.138666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:14.180403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1113/2195] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:14.191378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:14.193329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:14.244351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:14.275914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91994080-c9b6-44f2-a0ed-f6736cd1cbaa admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:14.280771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74005a08-76f9-4d06-894e-f33e7d613bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:14.281698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74005a08-76f9-4d06-894e-f33e7d613bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.282290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74005a08-76f9-4d06-894e-f33e7d613bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.283046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74005a08-76f9-4d06-894e-f33e7d613bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.300895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74005a08-76f9-4d06-894e-f33e7d613bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:14.302606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74005a08-76f9-4d06-894e-f33e7d613bf4 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:14.314712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1083/2196] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:14.617447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:14.618911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:14.633132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274b805b-c0a8-42a3-9806-b531351b30c2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:14.639615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:14.640694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.640988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.641267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] PATH_INFO: `/v3/projects/7cecdd3d4d754134b651cb151bb86f53` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.642487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cecdd3d4d754134b651cb151bb86f53)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:14.644601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:14.696246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cecdd3d4d754134b651cb151bb86f53', 'request_id': 'req-8faed5ee-4c39-480f-9593-1d9620592ae7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.701768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cecdd3d4d754134b651cb151bb86f53', 'request_id': 'req-8faed5ee-4c39-480f-9593-1d9620592ae7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.707739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cecdd3d4d754134b651cb151bb86f53', 'request_id': 'req-8faed5ee-4c39-480f-9593-1d9620592ae7'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.720507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] The token cache is being invalidate because project 7cecdd3d4d754134b651cb151bb86f53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:14.720791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.721407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8faed5ee-4c39-480f-9593-1d9620592ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:14.722985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1114/2197] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:14 2022] DELETE /identity/v3/projects/7cecdd3d4d754134b651cb151bb86f53 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:14.728630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.728928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.729207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.747491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:14.748748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:14.815523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1084/2198] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:14.817724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.818135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.818520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.841546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:14.842755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:14.851322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1115/2199] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:14.855510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:14.865042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b9ba0-7d65-4b89-b9f6-eded4c4d20fd None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:14.869585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:14.870233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.870518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.870792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.871554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:14.871862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:14.873438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:14.887576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1085/2200] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:14.891917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:14.901662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5066-9948-4ec3-a19e-84170eb25db3 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:14.906584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:14.907227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:14.907548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:14.907822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:14.908517 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:14.910454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:14.936043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1116/2201] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:14 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:14.940109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:14.950298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51580794-7e7b-4568-8e35-a89e6022845c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:15.015401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:15.016086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:15.016345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:15.016621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:15.017326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:15.019223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:15.039840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1086/2202] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:14 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:03:15.044430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:15.054867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6adf4f-48ac-4aa8-a95d-2f50f7778fdd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:15.059640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:15.060324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:15.060611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:15.060888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:15.061651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:15.063290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:15.074284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1117/2203] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:15.078236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:15.088297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7a208-561e-45d8-98ad-98208305eb0f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:15.093215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95d221c-3866-4692-bd4f-c5e87d468e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:15.093879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d221c-3866-4692-bd4f-c5e87d468e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:15.094169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d221c-3866-4692-bd4f-c5e87d468e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:15.094450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95d221c-3866-4692-bd4f-c5e87d468e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:15.095177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d221c-3866-4692-bd4f-c5e87d468e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:15.096806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95d221c-3866-4692-bd4f-c5e87d468e0e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:15.109040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1087/2204] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:15.113222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:15.124045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ae4af-67c0-4357-ab46-73806ec79ad1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:15.128643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:15.129518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:15.129806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:15.130086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] PATH_INFO: `/v3/projects/9c6274cbe443497e891b0b0c496a086d/users/8e052369e71d4d0a9de97284bf8d9eea/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:15.145237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c6274cbe443497e891b0b0c496a086d, user_id=8e052369e71d4d0a9de97284bf8d9eea, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:15.147934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:15.162328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1118/2205] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:15 2022] PUT /identity/v3/projects/9c6274cbe443497e891b0b0c496a086d/users/8e052369e71d4d0a9de97284bf8d9eea/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:15.173011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5416111-1fc2-4355-b72d-635a6dd7e80d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:15.173304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5416111-1fc2-4355-b72d-635a6dd7e80d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:15.173581 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5416111-1fc2-4355-b72d-635a6dd7e80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:15.201000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b5416111-1fc2-4355-b72d-635a6dd7e80d None None] MFA Rules not processed for user `8e052369e71d4d0a9de97284bf8d9eea`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:15.202166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b5416111-1fc2-4355-b72d-635a6dd7e80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:15.233554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:15.276739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1088/2206] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:15 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:15.288543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:15.290349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:15.348743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d405fcd3-d8f5-4b95-b85c-3ad4a62f1730 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:15.352413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:15.353064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:15.353347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:15.353624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:15.363118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:15.364573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:15.374539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1119/2207] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:15 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:18.336057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:18.336583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:18.336692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:18.356644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:18.357878 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:18.422158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1089/2208] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:03:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:18.436035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:18.437516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:18.443634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:18.518792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea686fbf-9357-4e60-a3b6-8ad956150642 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:18.525690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b69678a-ba1d-4f44-8557-e5ec9fc8e122 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:18.526419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69678a-ba1d-4f44-8557-e5ec9fc8e122 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:18.526705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69678a-ba1d-4f44-8557-e5ec9fc8e122 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:18.527061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b69678a-ba1d-4f44-8557-e5ec9fc8e122 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:18.536531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69678a-ba1d-4f44-8557-e5ec9fc8e122 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:18.538692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b69678a-ba1d-4f44-8557-e5ec9fc8e122 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:18.547785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1120/2209] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:03:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:03:27.119218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:27.232154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa5ecf-3878-4777-8c8d-859eb25a307e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:27.237256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:27.238662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:27.239316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:27.239600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] PATH_INFO: `/v3/users/8e052369e71d4d0a9de97284bf8d9eea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:27.240859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e052369e71d4d0a9de97284bf8d9eea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:27.243048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:27.336897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e052369e71d4d0a9de97284bf8d9eea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:27.347554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e052369e71d4d0a9de97284bf8d9eea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:27.348551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e052369e71d4d0a9de97284bf8d9eea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:27.349308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e052369e71d4d0a9de97284bf8d9eea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:27.366867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:27.372639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e052369e71d4d0a9de97284bf8d9eea'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:27.412120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1090/2210] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:27 2022] DELETE /identity/v3/users/8e052369e71d4d0a9de97284bf8d9eea => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:27.416295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:27.416835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:27.417108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:27.442879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:27.450151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:27.575407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1121/2211] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:27.584919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:27.586332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:27.661442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe3c4d-91a5-4a4e-9ba4-c46b25dfb581 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:27.668981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:27.670041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:27.670559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:27.671035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:27.692401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:27.696801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:27.712160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1091/2212] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:03:28.227801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:28.229728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:28.253786 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d175e-e37c-4f29-a019-9d86bab025dd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:28.263285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:28.266629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:28.267777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:28.268097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] PATH_INFO: `/v3/projects/9c6274cbe443497e891b0b0c496a086d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:28.270435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c6274cbe443497e891b0b0c496a086d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:28.273466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:28.387655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c6274cbe443497e891b0b0c496a086d', 'request_id': 'req-f4f5760a-ba6c-4042-9416-d9fd576cc281'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:28.397732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c6274cbe443497e891b0b0c496a086d', 'request_id': 'req-f4f5760a-ba6c-4042-9416-d9fd576cc281'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:28.409747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c6274cbe443497e891b0b0c496a086d', 'request_id': 'req-f4f5760a-ba6c-4042-9416-d9fd576cc281'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:28.431440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] The token cache is being invalidate because project 9c6274cbe443497e891b0b0c496a086d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:28.432426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:28.433272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f4f5760a-ba6c-4042-9416-d9fd576cc281 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:28.435188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1122/2213] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:28 2022] DELETE /identity/v3/projects/9c6274cbe443497e891b0b0c496a086d => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:32.393309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:32.397587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:32.413840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e0f4a2-c6ac-4b99-9190-e61de35b2c9e service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:32.527671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:32.528475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:32.528783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:32.529061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] PATH_INFO: `/v3/users/9abe16bf2cf941bf89cbecb889fe1e64` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:32.530361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abe16bf2cf941bf89cbecb889fe1e64)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:32.532438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:32.620443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abe16bf2cf941bf89cbecb889fe1e64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:32.629500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abe16bf2cf941bf89cbecb889fe1e64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:32.630565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abe16bf2cf941bf89cbecb889fe1e64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:32.632064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abe16bf2cf941bf89cbecb889fe1e64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:32.648420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abe16bf2cf941bf89cbecb889fe1e64'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:32.666232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1092/2214] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:32 2022] DELETE /identity/v3/users/9abe16bf2cf941bf89cbecb889fe1e64 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:32.670990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:32.671601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:32.671601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:32.700998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:32.702237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:32.715553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:32.814070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1123/2215] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:32.831690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:32.831690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:32.847476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:32.905551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd458427-0968-4906-b3f6-4632a2f7696c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:32.909393 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770d8d0-75a9-4a6a-bc1b-b6c534556ade service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:32.910055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d8d0-75a9-4a6a-bc1b-b6c534556ade service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:32.910341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d8d0-75a9-4a6a-bc1b-b6c534556ade service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:32.910628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770d8d0-75a9-4a6a-bc1b-b6c534556ade service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:32.921414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d8d0-75a9-4a6a-bc1b-b6c534556ade service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:32.922862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770d8d0-75a9-4a6a-bc1b-b6c534556ade service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:32.933742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1093/2216] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:33.281077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:33.282771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:33.303171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ef3a5b-c344-4c69-84e0-8f34cdbf7881 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:33.315835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:33.317035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:33.317412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:33.317794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] PATH_INFO: `/v3/projects/0f801e05e533408ca474ab40d649f7bf` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:33.319725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f801e05e533408ca474ab40d649f7bf)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:33.321822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:33.392642 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f801e05e533408ca474ab40d649f7bf', 'request_id': 'req-946596c2-462d-400b-9b0e-2c12e8636852'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:33.403900 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f801e05e533408ca474ab40d649f7bf', 'request_id': 'req-946596c2-462d-400b-9b0e-2c12e8636852'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:33.411587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f801e05e533408ca474ab40d649f7bf', 'request_id': 'req-946596c2-462d-400b-9b0e-2c12e8636852'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:33.429239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] The token cache is being invalidate because project 0f801e05e533408ca474ab40d649f7bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:33.429935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:33.430867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:33.433133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1124/2217] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:33 2022] DELETE /identity/v3/projects/0f801e05e533408ca474ab40d649f7bf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:42.928962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=21 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:43.425399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:50.663732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05fc78-174d-4d32-9c4d-18cc937769dc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:50.664198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05fc78-174d-4d32-9c4d-18cc937769dc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:50.664302 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05fc78-174d-4d32-9c4d-18cc937769dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:50.683518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0a05fc78-174d-4d32-9c4d-18cc937769dc None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:50.684736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0a05fc78-174d-4d32-9c4d-18cc937769dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:50.738365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1094/2218] 10.222.0.45 () {58 vars in 967 bytes} [Mon Jun 20 06:03:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:50.751573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:50.752982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:50.868023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946596c2-462d-400b-9b0e-2c12e8636852 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:50.873631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:50.874338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:50.877295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:50.877295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:50.888122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:50.889998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:50.900137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1125/2219] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:03:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:03:52.144060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:52.144546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:52.144647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:52.164942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:52.166142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:52.174815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1095/2220] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:03:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:52.187222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:52.197570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82083656-ce73-48e5-8400-19fcfdce6fc0 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:52.203112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:52.203827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:52.204108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:52.204475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:52.213351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:52.214926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:52.225411 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1126/2221] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:03:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:03:53.445775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:53.447357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:53.544497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936c0b9-b40c-4cc6-aaa3-13baad1c266e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:53.550165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:53.551149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:53.551550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:53.551952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] PATH_INFO: `/v3/users/5fcb569f32384f8d9d1878233d2732d6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:53.553865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fcb569f32384f8d9d1878233d2732d6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:53.556033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:53.634615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fcb569f32384f8d9d1878233d2732d6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:53.646772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fcb569f32384f8d9d1878233d2732d6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:53.647518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fcb569f32384f8d9d1878233d2732d6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:53.648564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fcb569f32384f8d9d1878233d2732d6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:53.665497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8573ab86-d187-4dfe-b2ff-dad8450debc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fcb569f32384f8d9d1878233d2732d6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:53.688631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1096/2222] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:53 2022] DELETE /identity/v3/users/5fcb569f32384f8d9d1878233d2732d6 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:53.988200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:53.999731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cc935e-e336-4f54-9ed7-e54386abd173 service swift] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:54.141032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:54.142049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.142399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.142625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] PATH_INFO: `/v3/projects/aaa8bf7b87244ac8a3c488fec16d54ca` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.143942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaa8bf7b87244ac8a3c488fec16d54ca)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:54.147176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:54.211016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaa8bf7b87244ac8a3c488fec16d54ca', 'request_id': 'req-3842844d-c53b-4724-a2f8-1cdaaef0be1c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.220567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaa8bf7b87244ac8a3c488fec16d54ca', 'request_id': 'req-3842844d-c53b-4724-a2f8-1cdaaef0be1c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.228037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaa8bf7b87244ac8a3c488fec16d54ca', 'request_id': 'req-3842844d-c53b-4724-a2f8-1cdaaef0be1c'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.254299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] The token cache is being invalidate because project aaa8bf7b87244ac8a3c488fec16d54ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:54.254724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.256919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.262969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1127/2223] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:53 2022] DELETE /identity/v3/projects/aaa8bf7b87244ac8a3c488fec16d54ca => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:54.265847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb025a-af73-46b4-b049-df2b440fa215 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.266313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb025a-af73-46b4-b049-df2b440fa215 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.266968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb025a-af73-46b4-b049-df2b440fa215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.275585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:54.278009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:54.291499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c4fb025a-af73-46b4-b049-df2b440fa215 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:54.295499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:54.298197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c4fb025a-af73-46b4-b049-df2b440fa215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:54.343937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:54.392974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842844d-c53b-4724-a2f8-1cdaaef0be1c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:54.407914 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:54.411567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.412531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.413024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] PATH_INFO: `/v3/users/455a5bf6e4e34afcaebdde3beaa619ea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.414720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=455a5bf6e4e34afcaebdde3beaa619ea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:54.416694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:54.419525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1097/2224] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:54.421618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.421920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.422203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.468371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:54.469649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:54.493601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1098/2225] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:54.493946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:54.533725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf8d57-de21-4c11-b49d-ce3f3def2eea None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:54.541420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '455a5bf6e4e34afcaebdde3beaa619ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.544933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:54.545747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.546160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.546568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.547541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:54.548071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:54.552579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '455a5bf6e4e34afcaebdde3beaa619ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.553530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '455a5bf6e4e34afcaebdde3beaa619ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.554390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '455a5bf6e4e34afcaebdde3beaa619ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.558594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:54.572079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b2174eba-82cc-4560-a9bd-08685b6679a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '455a5bf6e4e34afcaebdde3beaa619ea'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:54.589938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1099/2226] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:54.591486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:54.594702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:54.595342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1128/2227] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:54 2022] DELETE /identity/v3/users/455a5bf6e4e34afcaebdde3beaa619ea => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:03:54.601244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.601871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.602336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.626571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11b6ff0-8f69-4d52-9c8f-a01905eda3d0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:54.629140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:54.632976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:54.735136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1129/2228] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:54.744643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:54.746311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:54.790851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:54.792253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.793552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.794056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.795623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:54.802725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:54.810339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:54.851613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1100/2229] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:54 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:54.855328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:54.859007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e323719-e035-41d9-bdb6-a7f12b197ab4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:54.926179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:54.927062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.927513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.927895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.938163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e96f93-b55b-477b-8924-8603c89403af admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:54.942270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:54.944359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:54.944667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:54.945042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:54.945383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:54.945668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:54.946369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:54.948361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:54.955456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1130/2230] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:03:54.990973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1101/2231] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:54 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:54.992441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:55.009970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0170fcb-1224-47cf-b1e8-7575ab3749e2 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:55.019773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:55.020470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:55.021361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:55.021572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:55.022568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:55.025152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:55.041647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1131/2232] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:54 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:55.042906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:55.055142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4d1d4c-6d06-4a33-8da6-265154af3a33 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:55.062279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37323d1c-1821-4709-b7dd-6a51c165f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:55.065967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37323d1c-1821-4709-b7dd-6a51c165f9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:55.065967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37323d1c-1821-4709-b7dd-6a51c165f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:55.065967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37323d1c-1821-4709-b7dd-6a51c165f9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:55.065967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37323d1c-1821-4709-b7dd-6a51c165f9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:55.068041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37323d1c-1821-4709-b7dd-6a51c165f9ae admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:55.082035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1102/2233] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:55.085906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:55.102691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da2b826-4ca3-49de-beee-3128ea191e2c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:55.110551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:55.110551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:55.110551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:55.110551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] PATH_INFO: `/v3/projects/277d86f5a2954d5dad1d7ac8fc3b23b0/users/f32120c588564cc78b8c25cbb5301104/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:55.134865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] RBAC: Authorizing `identity:create_grant(project_id=277d86f5a2954d5dad1d7ac8fc3b23b0, user_id=f32120c588564cc78b8c25cbb5301104, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:55.138067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:55.153728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1132/2234] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:55 2022] PUT /identity/v3/projects/277d86f5a2954d5dad1d7ac8fc3b23b0/users/f32120c588564cc78b8c25cbb5301104/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:55.164889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:55.165269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:55.165559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:55.179504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:55.232030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] MFA Rules not processed for user `f32120c588564cc78b8c25cbb5301104`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:55.232707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:55.307750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1103/2235] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:55.319359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:55.321620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:55.397775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77120f0b-6397-4ad2-ac12-32f950f42110 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:55.401858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:55.402843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:55.403358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:55.403825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:55.418227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:55.419737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:55.432694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1133/2236] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:55.501157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:55.503830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:55.724495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96110b-f703-421d-bfc6-73d18a7d1099 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:55.730799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:55.732086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:55.732439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:55.732825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] PATH_INFO: `/v3/projects/8bdd02d892a24a40b4e17a4063a08765` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:55.734599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bdd02d892a24a40b4e17a4063a08765)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:55.736807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:55.832995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bdd02d892a24a40b4e17a4063a08765', 'request_id': 'req-0c9535dd-570e-46aa-ae90-b0517373df85'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:55.844307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bdd02d892a24a40b4e17a4063a08765', 'request_id': 'req-0c9535dd-570e-46aa-ae90-b0517373df85'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:55.851903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bdd02d892a24a40b4e17a4063a08765', 'request_id': 'req-0c9535dd-570e-46aa-ae90-b0517373df85'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:55.884729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] The token cache is being invalidate because project 8bdd02d892a24a40b4e17a4063a08765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:55.884729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:55.884729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:55.884729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1104/2237] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:55 2022] DELETE /identity/v3/projects/8bdd02d892a24a40b4e17a4063a08765 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:55.885737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:55.903812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d0228e-714b-4e12-9628-1fac513e06f5 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:55.999762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:56.015067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:56.016165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:56.016545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:56.016919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] PATH_INFO: `/v3/users/b5d7cb9dab374be0bfc9838a1042ebba` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:56.019203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d7cb9dab374be0bfc9838a1042ebba)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:56.023009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:56.096188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d7cb9dab374be0bfc9838a1042ebba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.104560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d7cb9dab374be0bfc9838a1042ebba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.105331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d7cb9dab374be0bfc9838a1042ebba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.105963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d7cb9dab374be0bfc9838a1042ebba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.122477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-111d28c8-53b8-48c0-b69e-538544ae09f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d7cb9dab374be0bfc9838a1042ebba'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.145979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1134/2238] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:55 2022] DELETE /identity/v3/users/b5d7cb9dab374be0bfc9838a1042ebba => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:56.505374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:56.507773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:56.535394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9535dd-570e-46aa-ae90-b0517373df85 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:56.562647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:56.612869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:56.613941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:56.614344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:56.615134 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] PATH_INFO: `/v3/projects/031d6f9897da43de9bc42bd52abc93fc` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:56.616480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031d6f9897da43de9bc42bd52abc93fc)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:56.618815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:56.677263 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031d6f9897da43de9bc42bd52abc93fc', 'request_id': 'req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.686718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031d6f9897da43de9bc42bd52abc93fc', 'request_id': 'req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.696489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031d6f9897da43de9bc42bd52abc93fc', 'request_id': 'req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.711497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] The token cache is being invalidate because project 031d6f9897da43de9bc42bd52abc93fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:56.711838 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.712965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2f3b4f80-aeb1-4b80-a6ec-4dbae14d85c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:56.714523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1105/2239] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:56 2022] DELETE /identity/v3/projects/031d6f9897da43de9bc42bd52abc93fc => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:56.719701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:56.719993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:56.720266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:56.741710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:56.743359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:56.816049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1135/2240] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:56.825523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:56.825523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:56.825523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:56.849451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:56.851080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:56.860149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1106/2241] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:56.864368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:56.876884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab539a-ecc4-4053-b99e-38bee16786cc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:56.883041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:56.883766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:56.884056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:56.884338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:56.885081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:56.885398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:56.887020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:56.900891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1136/2242] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:56.905097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:56.915400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b6fff9-2d7d-49c8-a408-c803144c6254 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:56.921564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:56.922334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:56.922706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:56.922821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:56.923569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:56.925518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:56.974906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1107/2243] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:56 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:56.976060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:56.988734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e34eae-0adb-4e2a-86a3-9fa1e4af12f5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.112297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.113086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.113474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.113851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.114775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.117135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.125591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:57.147756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1137/2244] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:56 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:03:57.151863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.167737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef59312-d7a8-4a22-8fce-599d08b48f4a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.173937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.174640 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.174916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.175198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.176116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.177797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.198059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1108/2245] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:57.201818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.217280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c423b199-399b-4806-b174-4f06da313edb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.222487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdea39-12ff-4433-ac7d-fee414960811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.223318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdea39-12ff-4433-ac7d-fee414960811 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.223692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdea39-12ff-4433-ac7d-fee414960811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.224017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdea39-12ff-4433-ac7d-fee414960811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.225027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdea39-12ff-4433-ac7d-fee414960811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.226460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdea39-12ff-4433-ac7d-fee414960811 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.262317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1138/2246] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:57.266314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.279679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bbff7-c2c6-4b9a-80d5-35df4e3102a2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.285188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.286193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.286566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.286914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] PATH_INFO: `/v3/projects/6647d0e4184f4e8eaa3e3aef02bf5c3c/users/a0f848d1eaf4404f80bfbfde15abcd30/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.313303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6647d0e4184f4e8eaa3e3aef02bf5c3c, user_id=a0f848d1eaf4404f80bfbfde15abcd30, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.318876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.343871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1109/2247] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:57 2022] PUT /identity/v3/projects/6647d0e4184f4e8eaa3e3aef02bf5c3c/users/a0f848d1eaf4404f80bfbfde15abcd30/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:03:57.355979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.356920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.356920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.420602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] MFA Rules not processed for user `a0f848d1eaf4404f80bfbfde15abcd30`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:57.422219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:57.554513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1139/2248] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:57.558719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.648968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:57.690789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f6aa4-a3a7-42ea-88a9-59ccb971be59 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.696409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.697458 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.697975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.698433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.699161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.701737 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.737985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1110/2249] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:57 2022] POST /identity/v3/projects => generated 377 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:57.741966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.757224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9cb553-7a47-4c78-b80a-35e28648e1b2 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.845852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.846816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.847198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.847600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.848393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.850445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.881745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1140/2250] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:03:57 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:03:57.882983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.895123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7b7a9-d939-4ed8-b3fa-f45de8c9f301 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.903113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.903916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.904273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.904625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.905439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.907446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:57.922063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1111/2251] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:57 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:57.927192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:57.946240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f707a-7d34-431d-9fae-14e6a8eb48e4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:57.959493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:57.960771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:57.961243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:57.961607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] PATH_INFO: `/v3/projects/9b1672c3347541f99c8c962d6b410895/users/2b2dee33a799407fb4555264c10be535/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:57.991857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1672c3347541f99c8c962d6b410895, user_id=2b2dee33a799407fb4555264c10be535, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:57.996025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:58.022826 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1141/2252] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:57 2022] PUT /identity/v3/projects/9b1672c3347541f99c8c962d6b410895/users/2b2dee33a799407fb4555264c10be535/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:58.027592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:58.054197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acfede0-9d62-437c-8b8e-e1eae519049b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:58.131475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d9635-6627-4a57-b533-d71a8167d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:58.132863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9635-6627-4a57-b533-d71a8167d9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.133428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9635-6627-4a57-b533-d71a8167d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.133811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d9635-6627-4a57-b533-d71a8167d9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.135664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9635-6627-4a57-b533-d71a8167d9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:58.137651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d9635-6627-4a57-b533-d71a8167d9ea admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:58.162543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:58.164803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1112/2253] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:03:58.185252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b142a7c-9f1f-43da-8a00-a05f9a95b95b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:58.194071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:58.196047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.196460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.197346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] PATH_INFO: `/v3/projects/9b1672c3347541f99c8c962d6b410895/users/2b2dee33a799407fb4555264c10be535/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.200301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1672c3347541f99c8c962d6b410895, user_id=2b2dee33a799407fb4555264c10be535, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:58.204700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:58.229102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1142/2254] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:03:58 2022] PUT /identity/v3/projects/9b1672c3347541f99c8c962d6b410895/users/2b2dee33a799407fb4555264c10be535/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:58.242160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.242491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.242776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.277764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] MFA Rules not processed for user `2b2dee33a799407fb4555264c10be535`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:58.278989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:58.344324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:58.346022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:58.366329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1113/2255] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:58 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:58.376709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:58.378365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:58.378630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:58.446932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:58.449156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d45fbd-42bb-4da8-a77c-b122812f5f48 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:58.456659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:58.457575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.457860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.458796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] PATH_INFO: `/v3/users/f32120c588564cc78b8c25cbb5301104` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.460165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32120c588564cc78b8c25cbb5301104)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:58.462278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:58.470078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0a61bb-d46c-4402-947d-0e8bb0fcca8b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:58.484001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:58.485805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.486089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.486368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.532091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32120c588564cc78b8c25cbb5301104'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:58.546778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32120c588564cc78b8c25cbb5301104'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:58.547858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32120c588564cc78b8c25cbb5301104'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:58.548795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32120c588564cc78b8c25cbb5301104'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:58.569206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a04ce7cc-693b-47bf-a9e1-9cd7d1a3d341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32120c588564cc78b8c25cbb5301104'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:58.587766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1143/2256] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:03:58 2022] DELETE /identity/v3/users/f32120c588564cc78b8c25cbb5301104 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:03:58.591570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:58.593127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:58.593447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.594048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.594944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.604469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1114/2257] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:03:58.618430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:58.619639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:58.749079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1144/2258] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:03:58.763670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:58.766466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:58.880941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8017821-58a6-40da-9495-493eacd18d22 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:58.951886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4a81b-0cb0-4a09-9c27-61effaeff42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:58.952387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4a81b-0cb0-4a09-9c27-61effaeff42b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:58.952849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4a81b-0cb0-4a09-9c27-61effaeff42b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:58.953325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4a81b-0cb0-4a09-9c27-61effaeff42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:58.986476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4a81b-0cb0-4a09-9c27-61effaeff42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:58.988312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4a81b-0cb0-4a09-9c27-61effaeff42b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:59.003880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1115/2259] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:03:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:03:59.387428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:59.389481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:59.414420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cf7d9-957e-4209-9982-37deb9f90633 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:59.429310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:59.434588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:59.438993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:59.439742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] PATH_INFO: `/v3/projects/277d86f5a2954d5dad1d7ac8fc3b23b0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:59.443923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=277d86f5a2954d5dad1d7ac8fc3b23b0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:59.450442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:59.564084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:59.585129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '277d86f5a2954d5dad1d7ac8fc3b23b0', 'request_id': 'req-4200fc99-cec8-4579-8c82-302128a91201'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:59.592447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '277d86f5a2954d5dad1d7ac8fc3b23b0', 'request_id': 'req-4200fc99-cec8-4579-8c82-302128a91201'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:59.602191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '277d86f5a2954d5dad1d7ac8fc3b23b0', 'request_id': 'req-4200fc99-cec8-4579-8c82-302128a91201'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:59.619867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] The token cache is being invalidate because project 277d86f5a2954d5dad1d7ac8fc3b23b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:03:59.620866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:59.623124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4200fc99-cec8-4579-8c82-302128a91201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:03:59.625186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1145/2260] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:03:59 2022] DELETE /identity/v3/projects/277d86f5a2954d5dad1d7ac8fc3b23b0 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:03:59.630151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:59.630657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:59.630758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:59.663485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:59.664765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:59.731322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:03:59.753981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1116/2261] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:59.763872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:59.764218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:59.764512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:59.791003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:03:59.792406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:03:59.802561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1146/2262] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:03:59.807225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:59.822987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242e594-e89c-4162-a3ed-a9f878c42af9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:59.880508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:59.880508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:59.880508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:59.880508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:59.882132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:03:59.882132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:59.882132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:03:59.936337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1117/2263] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:03:59.940484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:03:59.956708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b433b799-ef0e-44a7-b654-ae457533a6f0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:03:59.980361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:03:59.980584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:03:59.980886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:03:59.981089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:03:59.982021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:03:59.983760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:00.077457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:00.078111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1147/2264] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:03:59 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:00.106648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e19c4f3-de8e-4a81-ac7b-34df123a8150 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:00.185964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:00.186850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:00.187438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:00.187822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:00.188753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:00.190937 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:00.232654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1118/2265] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:04:00 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:00.237754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:00.254164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc942620-b3d7-4d93-897e-431ddb5c4e2a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:00.276381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:00.276946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:00.277234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:00.277511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:00.278263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:00.281347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:00.305658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1148/2266] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:00.310131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:00.325601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbef03f-443b-4182-bb0a-b72970a11c28 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:00.338413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765777d0-c0ab-4877-b5db-5f6573e93870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:00.339719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765777d0-c0ab-4877-b5db-5f6573e93870 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:00.340069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765777d0-c0ab-4877-b5db-5f6573e93870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:00.341137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765777d0-c0ab-4877-b5db-5f6573e93870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:00.342072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765777d0-c0ab-4877-b5db-5f6573e93870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:00.345040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765777d0-c0ab-4877-b5db-5f6573e93870 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:00.370190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1119/2267] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:00.371372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:00.386602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ddd41a-eb7f-4e90-bc22-3a01f91b54db admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:00.395081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:00.396868 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:00.397191 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:00.397567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] PATH_INFO: `/v3/projects/7c6ff8a12e78430f9b0171a87771b80b/users/2f1455920a81413a8264fbabba997da5/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:00.415637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c6ff8a12e78430f9b0171a87771b80b, user_id=2f1455920a81413a8264fbabba997da5, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:00.418504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:00.447002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1149/2268] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:00 2022] PUT /identity/v3/projects/7c6ff8a12e78430f9b0171a87771b80b/users/2f1455920a81413a8264fbabba997da5/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:00.452208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955f648-210a-4cdc-802e-268679af0f53 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:00.452565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955f648-210a-4cdc-802e-268679af0f53 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:00.452856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d955f648-210a-4cdc-802e-268679af0f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:00.480831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d955f648-210a-4cdc-802e-268679af0f53 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:00.480831 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d955f648-210a-4cdc-802e-268679af0f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:00.589841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1120/2269] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:00.599768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:00.602453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:00.689068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e358df-e790-4292-8bb9-9cfab4137e77 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:00.695873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:00.696890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:00.697459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:00.697844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:00.716206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:00.717830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:00.729821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1150/2270] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:01.775929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395114-c765-4322-a411-9dc8239c497c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:01.776479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395114-c765-4322-a411-9dc8239c497c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:01.777185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5395114-c765-4322-a411-9dc8239c497c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:01.797197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d5395114-c765-4322-a411-9dc8239c497c None None] MFA Rules not processed for user `ca882ae475a24e7f84124a1ab3e374f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:01.798584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d5395114-c765-4322-a411-9dc8239c497c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:01.920504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1121/2271] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:04.740301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:04.742741 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:04.811273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9711ad1d-040e-4482-969e-e0a1ec03153c service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:04.816105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:04.817267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:04.817783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:04.818531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:04.820811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:04.840622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:04.957863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:04.959450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:04.972945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1151/2272] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:04 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:07.687166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:07.687647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:07.687865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:07.700234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:07.735073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] MFA Rules not processed for user `2f1455920a81413a8264fbabba997da5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:07.737025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:07.826880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1122/2273] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:07.831713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:07.895555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3ee92-0fb3-46de-bb4f-5d0e394824fd service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:07.900315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:07.900984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:07.901269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:07.901545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:07.902267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:07.904198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:07.942648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1152/2274] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:04:07 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:07.946845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:07.958305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfc7c1a-62a6-4c3a-973b-82bcee6c5af6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:08.014175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:08.014875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:08.015165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:08.015480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:08.016204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:08.018179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:08.043601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1123/2275] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:04:07 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:08.047607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:08.058589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf182b3d-7389-49f8-addb-61c5941ce4fb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:08.063852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:08.064534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:08.064846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:08.065131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:08.065867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:08.067520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:08.078421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1153/2276] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:08.083187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:08.093517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915a1e2-e324-47a2-85e5-ca36bfe15716 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:08.098198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:08.099654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:08.099960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:08.100237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] PATH_INFO: `/v3/projects/4ccda0be70464b2dbb4d47c0ac4cfbac/users/b298d9d17f8d4fe9a409b18dcc4d5ebf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:08.114839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccda0be70464b2dbb4d47c0ac4cfbac, user_id=b298d9d17f8d4fe9a409b18dcc4d5ebf, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:08.117624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:08.131374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1124/2277] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:08 2022] PUT /identity/v3/projects/4ccda0be70464b2dbb4d47c0ac4cfbac/users/b298d9d17f8d4fe9a409b18dcc4d5ebf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:08.135330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:08.146259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ca9eb-2fc3-41ca-b9f4-7245557db7ff admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:08.214908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1bde0a-f9c9-4df2-bac2-0a903175b7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:08.214908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bde0a-f9c9-4df2-bac2-0a903175b7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:08.214908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bde0a-f9c9-4df2-bac2-0a903175b7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:08.214908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1bde0a-f9c9-4df2-bac2-0a903175b7ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:08.217201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bde0a-f9c9-4df2-bac2-0a903175b7ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:08.218713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1bde0a-f9c9-4df2-bac2-0a903175b7ea admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:08.229813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1154/2278] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:08.234077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:08.245013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9ab95-cb20-42b0-955d-c572747c26b8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:08.249847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:08.250745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:08.251028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:08.251323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] PATH_INFO: `/v3/projects/4ccda0be70464b2dbb4d47c0ac4cfbac/users/b298d9d17f8d4fe9a409b18dcc4d5ebf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:08.253516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccda0be70464b2dbb4d47c0ac4cfbac, user_id=b298d9d17f8d4fe9a409b18dcc4d5ebf, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:08.256770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:08.270702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1125/2279] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:08 2022] PUT /identity/v3/projects/4ccda0be70464b2dbb4d47c0ac4cfbac/users/b298d9d17f8d4fe9a409b18dcc4d5ebf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:04:13.211559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318213d-28f7-48d5-8991-a3830f7fc4c0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:13.212021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318213d-28f7-48d5-8991-a3830f7fc4c0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:13.212827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d318213d-28f7-48d5-8991-a3830f7fc4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:13.283543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d318213d-28f7-48d5-8991-a3830f7fc4c0 None None] MFA Rules not processed for user `b298d9d17f8d4fe9a409b18dcc4d5ebf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:13.311776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d318213d-28f7-48d5-8991-a3830f7fc4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:13.375429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:13.520781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1155/2280] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:13.521656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:13.523905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:13.578209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:13.659987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7003e09f-d9bb-4095-be64-30c6700b3e57 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:13.666705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:13.667492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:13.667841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:13.668185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:13.741791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:13.743372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:13.753344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1126/2281] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:04:13.898852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47085c8-ad2d-4fc7-9480-e3159bf8cba2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:13.899346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47085c8-ad2d-4fc7-9480-e3159bf8cba2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:13.899446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47085c8-ad2d-4fc7-9480-e3159bf8cba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:13.942699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a47085c8-ad2d-4fc7-9480-e3159bf8cba2 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:13.947917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a47085c8-ad2d-4fc7-9480-e3159bf8cba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:14.097469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1156/2282] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:04:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:04:14.121512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:14.122862 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:14.299295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafde995-7363-42ac-ba0e-e95f840bffbf service glance] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:14.319975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c6d1b-1e02-4547-bdef-0bcc9158de98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:14.320653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c6d1b-1e02-4547-bdef-0bcc9158de98 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:14.320944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c6d1b-1e02-4547-bdef-0bcc9158de98 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:14.321222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c6d1b-1e02-4547-bdef-0bcc9158de98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:14.363743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c6d1b-1e02-4547-bdef-0bcc9158de98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:14.369593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c6d1b-1e02-4547-bdef-0bcc9158de98 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:14.388671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1127/2283] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:04:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:04:23.348500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca5f2-f4f0-42ee-8fb8-4264fcdf5242 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:23.349057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca5f2-f4f0-42ee-8fb8-4264fcdf5242 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:23.349057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ca5f2-f4f0-42ee-8fb8-4264fcdf5242 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:23.352279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1157/2284] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:04:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:04:23.358383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb007e-ae42-427e-8dd0-2d5c55106de8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:23.358869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb007e-ae42-427e-8dd0-2d5c55106de8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:23.359002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb007e-ae42-427e-8dd0-2d5c55106de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:23.419126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e8eb007e-ae42-427e-8dd0-2d5c55106de8 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:23.421807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8eb007e-ae42-427e-8dd0-2d5c55106de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:23.486194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:23.538272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1128/2285] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:04:23.562487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ca5f2-f4f0-42ee-8fb8-4264fcdf5242 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:23.578047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ca5f2-f4f0-42ee-8fb8-4264fcdf5242 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:23.581857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291472c2-f1b5-4bbd-932e-d0fe9816869e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:23.582862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291472c2-f1b5-4bbd-932e-d0fe9816869e service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:23.583142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291472c2-f1b5-4bbd-932e-d0fe9816869e service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:23.583663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291472c2-f1b5-4bbd-932e-d0fe9816869e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:23.606526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291472c2-f1b5-4bbd-932e-d0fe9816869e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:23.608107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291472c2-f1b5-4bbd-932e-d0fe9816869e service glance] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:23.621683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1158/2286] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:23.706588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140bc7a-a8ab-4506-93c4-748e9422a1a6 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:23.706588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140bc7a-a8ab-4506-93c4-748e9422a1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:23.706898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140bc7a-a8ab-4506-93c4-748e9422a1a6 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:23.708985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1129/2287] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:04:23.714007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081447e-7edc-4750-bbcb-2fd0e4c909a7 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:23.714369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081447e-7edc-4750-bbcb-2fd0e4c909a7 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:23.714715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b081447e-7edc-4750-bbcb-2fd0e4c909a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:23.748250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b081447e-7edc-4750-bbcb-2fd0e4c909a7 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:23.749612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b081447e-7edc-4750-bbcb-2fd0e4c909a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:23.835414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1159/2288] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:04:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:23.844299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140bc7a-a8ab-4506-93c4-748e9422a1a6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:23.848691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d140bc7a-a8ab-4506-93c4-748e9422a1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:23.865538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140bc7a-a8ab-4506-93c4-748e9422a1a6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:23.871328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581ad12-8ce2-4700-8739-60ffa5da5ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:23.872201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581ad12-8ce2-4700-8739-60ffa5da5ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:23.872477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581ad12-8ce2-4700-8739-60ffa5da5ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:23.872828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581ad12-8ce2-4700-8739-60ffa5da5ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:23.882578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581ad12-8ce2-4700-8739-60ffa5da5ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:23.884313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b581ad12-8ce2-4700-8739-60ffa5da5ab4 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:23.896024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1130/2289] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:04:24.002832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db48ceb-4506-4f29-aebc-c9477dacfc06 None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:24.004280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db48ceb-4506-4f29-aebc-c9477dacfc06 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:24.004879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db48ceb-4506-4f29-aebc-c9477dacfc06 None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:24.007634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1160/2290] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:04:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:04:24.013446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc476c-01ef-428b-951b-fb571236f3c4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:24.016658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc476c-01ef-428b-951b-fb571236f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:24.017027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc476c-01ef-428b-951b-fb571236f3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:24.040672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-75dc476c-01ef-428b-951b-fb571236f3c4 None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:24.042005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-75dc476c-01ef-428b-951b-fb571236f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:24.058454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1131/2291] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:04:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:24.070437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db48ceb-4506-4f29-aebc-c9477dacfc06 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:24.084748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db48ceb-4506-4f29-aebc-c9477dacfc06 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:24.088974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1e4f25-9fcf-491c-82b5-353bb0643f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:24.089632 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4f25-9fcf-491c-82b5-353bb0643f9f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:24.089916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4f25-9fcf-491c-82b5-353bb0643f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:24.090447 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1e4f25-9fcf-491c-82b5-353bb0643f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:24.103715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e4f25-9fcf-491c-82b5-353bb0643f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:24.106929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1e4f25-9fcf-491c-82b5-353bb0643f9f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:24.133972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1161/2292] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:25.171999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:25.172537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:25.172537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:25.190273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] MFA Rules not processed for user `2f1455920a81413a8264fbabba997da5`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:25.191773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:25.199510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1132/2293] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:28.759562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:28.760205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:28.760443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:28.780823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:28.782077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:28.790889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1162/2294] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:04:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:28.812045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:28.828625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82920340-b0de-409c-93a5-9c5b3bc3c88d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:28.836732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a099858-cea4-457c-928c-f4f80d5a7fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:28.838959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a099858-cea4-457c-928c-f4f80d5a7fba service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:28.839506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a099858-cea4-457c-928c-f4f80d5a7fba service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:28.840035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a099858-cea4-457c-928c-f4f80d5a7fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:28.850987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a099858-cea4-457c-928c-f4f80d5a7fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:28.853232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a099858-cea4-457c-928c-f4f80d5a7fba service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:28.876765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1133/2295] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:04:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:04:38.775108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:38.860334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=33 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:39.589555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:39.591220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:39.656238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7009d46-b8c5-4c15-ac8c-0973abe7b581 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:39.661349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:39.662240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:39.662526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:39.662801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] PATH_INFO: `/v3/users/a0f848d1eaf4404f80bfbfde15abcd30` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:39.664034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0f848d1eaf4404f80bfbfde15abcd30)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:39.665952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:39.715766 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0f848d1eaf4404f80bfbfde15abcd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:39.723637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0f848d1eaf4404f80bfbfde15abcd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:39.724362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0f848d1eaf4404f80bfbfde15abcd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:39.724965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0f848d1eaf4404f80bfbfde15abcd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:39.737118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0f848d1eaf4404f80bfbfde15abcd30'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:39.756726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1163/2296] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:04:39 2022] DELETE /identity/v3/users/a0f848d1eaf4404f80bfbfde15abcd30 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:39.758235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:39.758530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:39.764057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:39.798447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:39.799925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:39.911001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1134/2297] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:39.928624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:39.930784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:40.015898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b5352-9008-41f8-9693-9434ff21be1e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:40.021313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:40.021991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:40.022281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:40.022563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:40.034677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:40.036630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:40.051827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1164/2298] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:40.424488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:40.426029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:40.444878 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbf225-5e76-4db7-b4c8-527c918cfb33 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:40.450084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:40.450987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:40.451314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:40.451533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] PATH_INFO: `/v3/projects/6647d0e4184f4e8eaa3e3aef02bf5c3c` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:40.452736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6647d0e4184f4e8eaa3e3aef02bf5c3c)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:40.454648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:40.501316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6647d0e4184f4e8eaa3e3aef02bf5c3c', 'request_id': 'req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.509291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6647d0e4184f4e8eaa3e3aef02bf5c3c', 'request_id': 'req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.515654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6647d0e4184f4e8eaa3e3aef02bf5c3c', 'request_id': 'req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.528909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] The token cache is being invalidate because project 6647d0e4184f4e8eaa3e3aef02bf5c3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:04:40.529336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.529961 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.531565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1135/2299] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:04:40 2022] DELETE /identity/v3/projects/6647d0e4184f4e8eaa3e3aef02bf5c3c => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:40.535461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:40.545577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68400a39-34b0-4164-910e-2993ff256ec0 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:40.563737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:40.605759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:40.606667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:40.607061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:40.607381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] PATH_INFO: `/v3/users/2b2dee33a799407fb4555264c10be535` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:40.608606 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2dee33a799407fb4555264c10be535)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:40.610523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:40.660367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2dee33a799407fb4555264c10be535'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.667144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2dee33a799407fb4555264c10be535'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.667889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2dee33a799407fb4555264c10be535'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.668495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2dee33a799407fb4555264c10be535'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.680417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6fee2520-ff92-422a-831e-938f7b0145be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2dee33a799407fb4555264c10be535'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:40.693814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1165/2300] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:04:40 2022] DELETE /identity/v3/users/2b2dee33a799407fb4555264c10be535 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:40.969093 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:40.970544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:40.995503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a0fff3-d6fc-4750-b8fb-ff6403a8c28e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.055411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.056340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.056628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.056908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] PATH_INFO: `/v3/projects/9b1672c3347541f99c8c962d6b410895` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.058950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1672c3347541f99c8c962d6b410895)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.060907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.083115 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:41.112982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1672c3347541f99c8c962d6b410895', 'request_id': 'req-a7270f36-cce7-40a2-8f80-5493bd0ea28f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:41.118650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1672c3347541f99c8c962d6b410895', 'request_id': 'req-a7270f36-cce7-40a2-8f80-5493bd0ea28f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:41.124868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1672c3347541f99c8c962d6b410895', 'request_id': 'req-a7270f36-cce7-40a2-8f80-5493bd0ea28f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:41.138304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] The token cache is being invalidate because project 9b1672c3347541f99c8c962d6b410895 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:04:41.138597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:41.139234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-a7270f36-cce7-40a2-8f80-5493bd0ea28f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:41.140816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1136/2301] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:04:40 2022] DELETE /identity/v3/projects/9b1672c3347541f99c8c962d6b410895 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:41.146146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.146593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.146737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.182933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:41.184170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:41.244227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1166/2302] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:41.253459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.253750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.254033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.272576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:41.273795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:41.282078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1137/2303] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:41.286089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.295912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c50c69-6039-41e2-b5f6-5ef8e713ec4a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.300628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.301283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.301573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.301855 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.302587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:04:41.302898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.304559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.316975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1167/2304] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:04:41.321198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.330433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc86da37-1311-46ca-9f4b-88cd11059b63 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.337307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.337806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.338081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.338353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.340743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.341513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.372802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.373474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1138/2305] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:41.383755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c6550-4479-48b2-a210-8c3c8abfc3f0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.394265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:41.447059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.447859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.448143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.448420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.449109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.451053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.474993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1168/2306] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:41.476491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.488727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22988e96-4596-42d7-88e6-95af28d42970 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.494869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.499038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.507273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.507802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.523683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.525506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.547031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1139/2307] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:04:41.550762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.562459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6da0e-9c8a-4dc0-b112-f2275b5b21ad admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.568356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb4e9e-4c89-454b-9720-ae3f1f4c4798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.569174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb4e9e-4c89-454b-9720-ae3f1f4c4798 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.569559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb4e9e-4c89-454b-9720-ae3f1f4c4798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.570143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb4e9e-4c89-454b-9720-ae3f1f4c4798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.570988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb4e9e-4c89-454b-9720-ae3f1f4c4798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.572912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb4e9e-4c89-454b-9720-ae3f1f4c4798 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.585567 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1169/2308] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:41.593887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.613909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0470c4f5-b023-4aad-9f97-adb9998e38a1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.623197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.623197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.628683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.628683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] PATH_INFO: `/v3/projects/d69dbcf0f6ac4aab9e71bf7363025944/users/ad521c6d17df4b5eb14cb899b2f196fc/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.643957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d69dbcf0f6ac4aab9e71bf7363025944, user_id=ad521c6d17df4b5eb14cb899b2f196fc, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.646673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.667325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1140/2309] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:41 2022] PUT /identity/v3/projects/d69dbcf0f6ac4aab9e71bf7363025944/users/ad521c6d17df4b5eb14cb899b2f196fc/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:41.673031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.673357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.673645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.702602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] MFA Rules not processed for user `ad521c6d17df4b5eb14cb899b2f196fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:41.703858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:41.774349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1170/2310] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:41.778667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.855571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251efdbc-4698-4e34-be06-218bd77721f5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.859996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.860651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.860935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.861209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.861897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.864374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.892916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1141/2311] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/projects => generated 391 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:41.894390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:41.905965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc9fae-112e-45d7-b687-ceb30864d8eb None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:41.965618 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:41.966308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:41.966590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:41.966948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:41.967604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:41.969579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:41.991075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1171/2312] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:04:41 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:41.995456 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:42.006301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96555e7-e4e6-4a16-91eb-41e0bcd6511e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:42.011198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:42.011915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.012200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.012478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.013210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:42.014827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:42.026313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1142/2313] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:42.030711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:42.041637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde7bc6-7b26-4d37-b388-08695a311345 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:42.046296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:42.047204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.047516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.047793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] PATH_INFO: `/v3/projects/3c06d254857b40d1bf3565d133d0f10f/users/19ae8a5eeec9491bb14ccd94a1606b91/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.057579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:42.067572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c06d254857b40d1bf3565d133d0f10f, user_id=19ae8a5eeec9491bb14ccd94a1606b91, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:42.070364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:42.084492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1172/2314] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:42 2022] PUT /identity/v3/projects/3c06d254857b40d1bf3565d133d0f10f/users/19ae8a5eeec9491bb14ccd94a1606b91/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:04:42.088716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:42.099969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f70c6ab-99d2-4369-a111-2c5877b1402e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:42.113538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:42.170646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd9b91-b67f-44da-86a8-27d2a688c464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:42.171452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd9b91-b67f-44da-86a8-27d2a688c464 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.171808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd9b91-b67f-44da-86a8-27d2a688c464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.172182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd9b91-b67f-44da-86a8-27d2a688c464 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.173139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd9b91-b67f-44da-86a8-27d2a688c464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:42.174845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd9b91-b67f-44da-86a8-27d2a688c464 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:42.186308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1143/2315] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:04:42.190333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:42.203011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621cf0cf-7e58-4f50-8c7a-fb8024c25eb9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:42.208176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:42.209257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.209670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.210232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] PATH_INFO: `/v3/projects/3c06d254857b40d1bf3565d133d0f10f/users/19ae8a5eeec9491bb14ccd94a1606b91/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.212443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c06d254857b40d1bf3565d133d0f10f, user_id=19ae8a5eeec9491bb14ccd94a1606b91, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:42.215491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:42.227209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1173/2316] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:42 2022] PUT /identity/v3/projects/3c06d254857b40d1bf3565d133d0f10f/users/19ae8a5eeec9491bb14ccd94a1606b91/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:42.237841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.238151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.238438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.265363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] MFA Rules not processed for user `19ae8a5eeec9491bb14ccd94a1606b91`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:42.266693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:42.341527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1144/2317] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:42.352997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:42.361535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:42.476283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70948a-6690-4fb1-869f-121cfe7b59d7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:42.480535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:42.481369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.481797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.482176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.493062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:42.495013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:42.516119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1174/2318] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:42.576757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:42.578476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:42.593631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f4c5db-d222-49db-8863-9472602c5a73 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:42.598584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea22172-78fa-4032-ae73-07ddb9731710 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:42.600163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22172-78fa-4032-ae73-07ddb9731710 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:42.600455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22172-78fa-4032-ae73-07ddb9731710 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:42.600741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea22172-78fa-4032-ae73-07ddb9731710 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:42.675203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea22172-78fa-4032-ae73-07ddb9731710 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:42.676775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea22172-78fa-4032-ae73-07ddb9731710 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:42.689478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1145/2319] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:45.374115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:45.376289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:45.491052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64cf097-f014-4be9-8de4-b094288d0aff service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:45.497253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:45.498229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:45.498753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:45.499821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:45.502268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:45.515739 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:45.517788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:45.532222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1175/2320] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:45.785465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:45.785988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:45.786082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:45.804358 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:45.805688 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:45.845519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:45.875073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1146/2321] 10.222.0.45 () {58 vars in 1009 bytes} [Mon Jun 20 06:04:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:45.888147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:45.889584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:45.922097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:45.964924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c488c-9268-4a12-85c3-1a832d435fcf service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:45.970827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198a3d0-0ded-453c-b2cb-95141024e905 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:45.972346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198a3d0-0ded-453c-b2cb-95141024e905 service swift] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:45.972346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198a3d0-0ded-453c-b2cb-95141024e905 service swift] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:45.972346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198a3d0-0ded-453c-b2cb-95141024e905 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:45.981274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198a3d0-0ded-453c-b2cb-95141024e905 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:45.982859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198a3d0-0ded-453c-b2cb-95141024e905 service swift] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:45.991721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1176/2322] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:04:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:04:46.106636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:46.244745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0486e39-1c0e-4a28-b93b-87bda32779e5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:46.254288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:46.255704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:46.256762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:46.256762 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] PATH_INFO: `/v3/users/ad521c6d17df4b5eb14cb899b2f196fc` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:46.258237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad521c6d17df4b5eb14cb899b2f196fc)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:46.271150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:46.385794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad521c6d17df4b5eb14cb899b2f196fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:46.407909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad521c6d17df4b5eb14cb899b2f196fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:46.410396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad521c6d17df4b5eb14cb899b2f196fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:46.411157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad521c6d17df4b5eb14cb899b2f196fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:46.441749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad521c6d17df4b5eb14cb899b2f196fc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:46.480245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1147/2323] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:04:46 2022] DELETE /identity/v3/users/ad521c6d17df4b5eb14cb899b2f196fc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:46.484742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:46.485260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:46.485344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:46.513908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:46.515164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:46.662420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1177/2324] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:46.669272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:46.671607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:46.830161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfbaa7-7fe5-4ac9-b8e1-c058394a89b9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:46.836956 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:46.837741 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:46.838030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:46.838304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:46.861729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:46.864047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:46.895651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:46.918702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1148/2325] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:47.621891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:47.624932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:47.647862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6cdfe6-ec76-4628-b086-3cd8097b6ad4 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:47.652712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:47.653614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:47.653896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:47.654163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] PATH_INFO: `/v3/projects/d69dbcf0f6ac4aab9e71bf7363025944` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:47.655405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d69dbcf0f6ac4aab9e71bf7363025944)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:47.657315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:47.707405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd69dbcf0f6ac4aab9e71bf7363025944', 'request_id': 'req-47023490-844f-4817-89fc-2f247e90c46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.713207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd69dbcf0f6ac4aab9e71bf7363025944', 'request_id': 'req-47023490-844f-4817-89fc-2f247e90c46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.720381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd69dbcf0f6ac4aab9e71bf7363025944', 'request_id': 'req-47023490-844f-4817-89fc-2f247e90c46f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.740205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] The token cache is being invalidate because project d69dbcf0f6ac4aab9e71bf7363025944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:04:47.740598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.741153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.742956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1178/2326] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:04:47 2022] DELETE /identity/v3/projects/d69dbcf0f6ac4aab9e71bf7363025944 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:47.747536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:47.758167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58e76e-5896-4bcc-a785-1210d8aac717 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:47.819877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:47.820963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:47.821359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:47.821765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] PATH_INFO: `/v3/users/19ae8a5eeec9491bb14ccd94a1606b91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:47.823449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ae8a5eeec9491bb14ccd94a1606b91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:47.825703 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:47.883569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ae8a5eeec9491bb14ccd94a1606b91'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.890603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ae8a5eeec9491bb14ccd94a1606b91'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.891306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ae8a5eeec9491bb14ccd94a1606b91'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.891920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ae8a5eeec9491bb14ccd94a1606b91'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.905257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bfc0e19d-fc69-4397-ac14-0b1053da853e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ae8a5eeec9491bb14ccd94a1606b91'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:47.918175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1149/2327] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:04:47 2022] DELETE /identity/v3/users/19ae8a5eeec9491bb14ccd94a1606b91 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:48.266413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:48.268017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:48.291382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47023490-844f-4817-89fc-2f247e90c46f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:48.340357 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:48.360808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:48.361785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:48.362603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:48.362881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] PATH_INFO: `/v3/projects/3c06d254857b40d1bf3565d133d0f10f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:48.364102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c06d254857b40d1bf3565d133d0f10f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:48.366018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:48.414692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c06d254857b40d1bf3565d133d0f10f', 'request_id': 'req-e23a5874-8030-4793-8f9f-a1328f638827'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:48.420298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c06d254857b40d1bf3565d133d0f10f', 'request_id': 'req-e23a5874-8030-4793-8f9f-a1328f638827'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:48.430234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c06d254857b40d1bf3565d133d0f10f', 'request_id': 'req-e23a5874-8030-4793-8f9f-a1328f638827'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:48.445070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] The token cache is being invalidate because project 3c06d254857b40d1bf3565d133d0f10f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:04:48.445603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:48.446428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e23a5874-8030-4793-8f9f-a1328f638827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:04:48.448759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1179/2328] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:04:48 2022] DELETE /identity/v3/projects/3c06d254857b40d1bf3565d133d0f10f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:48.453497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:48.453968 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:48.454477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:48.473372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:48.474673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:48.565082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1150/2329] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:48.573563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:48.573956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:48.574339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:48.600645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:48.602211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:48.630899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1180/2330] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:04:48.635668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:48.650547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f0ec21-602c-4284-8c0d-8bd9f830b67d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:48.671301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:48.672753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:48.674226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:48.676600 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:48.678754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:04:48.679118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:48.684493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:48.720921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1151/2331] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:04:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:04:48.726273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:48.754597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f5e68-f933-452f-a300-4cf207ae13c0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:48.769933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:48.770941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:48.772790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:48.773908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:48.775968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:48.783677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:48.878367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1181/2332] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:04:48 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:48.888256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:48.895820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:48.900996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da5b075-7ad5-4fb4-8dd7-bd19645c809d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:49.004035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:49.005455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:49.005748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:49.006053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:49.006742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:49.008710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:49.032109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1152/2333] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:04:48 2022] POST /identity/v3/users => generated 288 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:04:49.037083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:49.048283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d98567-8b4e-4189-b3ad-304551a813e7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:49.053126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:49.053823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:49.054109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:49.054389 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:49.055115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:49.056783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:49.068164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1182/2334] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:04:49.072031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:49.084498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e727c56-3493-48f1-910c-b3f18d27aa87 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:49.091522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ab0477-3d19-4a92-a5bf-adae2f8cd98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:49.092451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0477-3d19-4a92-a5bf-adae2f8cd98c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:49.092933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0477-3d19-4a92-a5bf-adae2f8cd98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:49.093414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ab0477-3d19-4a92-a5bf-adae2f8cd98c admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:49.094379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0477-3d19-4a92-a5bf-adae2f8cd98c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:49.096702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ab0477-3d19-4a92-a5bf-adae2f8cd98c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:49.108848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1153/2335] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:04:49.113380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:49.126212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b12b209-75a6-417e-863b-7a5194d78553 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:49.131692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:49.132587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:49.132875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:49.133149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] PATH_INFO: `/v3/projects/85a4ee2516eb49de925517867805b754/users/7a00c487dd63469ca6710388c65ea418/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:49.154267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a4ee2516eb49de925517867805b754, user_id=7a00c487dd63469ca6710388c65ea418, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:49.165899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:49.198065 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1183/2336] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:04:49 2022] PUT /identity/v3/projects/85a4ee2516eb49de925517867805b754/users/7a00c487dd63469ca6710388c65ea418/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:04:49.208996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ad1ca-494a-484d-8a7c-ed73bcf55e08 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:49.209461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ad1ca-494a-484d-8a7c-ed73bcf55e08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:49.209872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ad1ca-494a-484d-8a7c-ed73bcf55e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:49.249182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3e6ad1ca-494a-484d-8a7c-ed73bcf55e08 None None] MFA Rules not processed for user `7a00c487dd63469ca6710388c65ea418`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:04:49.250439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3e6ad1ca-494a-484d-8a7c-ed73bcf55e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:49.350818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1154/2337] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:04:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:04:49.364446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:04:49.366678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:04:49.472938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daefc0-3cb5-4373-ab6d-06cc7a10fa8e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:04:49.480200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:04:49.480883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:04:49.481222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:04:49.481561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:04:49.513083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:04:49.515051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:04:49.526575 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1184/2338] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:04:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:04:59.346692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=64,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:04:59.522328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:05:07.450561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b6a67-2d19-4385-9591-b5bdf0c22ed1 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:07.451008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b6a67-2d19-4385-9591-b5bdf0c22ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:07.451121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b6a67-2d19-4385-9591-b5bdf0c22ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:07.468531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8e0b6a67-2d19-4385-9591-b5bdf0c22ed1 None None] MFA Rules not processed for user `6490c3d52b5a45aea9236e171ae94770`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:05:07.469740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8e0b6a67-2d19-4385-9591-b5bdf0c22ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:07.522597 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1155/2339] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:05:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:05:14.547671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:14.613768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50eeda-8dd8-4431-b269-d428f263c027 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:14.618605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:14.619530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:14.619822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:14.620103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] PATH_INFO: `/v3/users/7a00c487dd63469ca6710388c65ea418` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:14.621351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a00c487dd63469ca6710388c65ea418)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:14.623356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:14.670889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a00c487dd63469ca6710388c65ea418'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:14.677897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a00c487dd63469ca6710388c65ea418'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:14.678604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a00c487dd63469ca6710388c65ea418'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:14.680520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a00c487dd63469ca6710388c65ea418'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:14.693276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a00c487dd63469ca6710388c65ea418'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:14.707884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1185/2340] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:05:14 2022] DELETE /identity/v3/users/7a00c487dd63469ca6710388c65ea418 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:05:14.712942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:14.713398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:14.713493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:14.731298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:05:14.732546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:14.790729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1156/2341] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:05:14.800268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:14.801699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:14.861284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff44966-6fb6-496c-aaa9-88e2b8fc1cf4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:14.864921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c21d81-f409-46b3-b2fb-c1fa79794a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:14.865570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21d81-f409-46b3-b2fb-c1fa79794a64 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:14.865860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21d81-f409-46b3-b2fb-c1fa79794a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:14.866141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c21d81-f409-46b3-b2fb-c1fa79794a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:14.875076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21d81-f409-46b3-b2fb-c1fa79794a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:14.876570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c21d81-f409-46b3-b2fb-c1fa79794a64 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:14.886470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1186/2342] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:05:15.232012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.233510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:15.246723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb86d29-fc74-4978-b673-bbee1120536e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.252219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.253100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.253448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.253733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] PATH_INFO: `/v3/projects/85a4ee2516eb49de925517867805b754` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.254950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a4ee2516eb49de925517867805b754)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.256941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.315485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a4ee2516eb49de925517867805b754', 'request_id': 'req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:15.321035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a4ee2516eb49de925517867805b754', 'request_id': 'req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:15.328198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a4ee2516eb49de925517867805b754', 'request_id': 'req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:15.341329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] The token cache is being invalidate because project 85a4ee2516eb49de925517867805b754 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:05:15.341806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:15.342720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-61eec8d7-4a03-45ac-91d3-82dd9239d5ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:05:15.344429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1157/2343] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:05:15 2022] DELETE /identity/v3/projects/85a4ee2516eb49de925517867805b754 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:05:15.350437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.350825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.351182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.370025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:05:15.371170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:15.387615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:05:15.430398 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1187/2344] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:05:15.439140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.439464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.439745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.458082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:05:15.459299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:15.467676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1158/2345] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:05:15.472378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.482087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b070c-ae11-4a62-9544-53da55e29a40 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.486704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.487395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.487682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.487963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.488691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:05:15.489005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.490610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.501831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1188/2346] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:05:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:05:15.505861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.515178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45039cd6-bee4-4de6-9c4c-c2f73e1af320 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.520555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.521241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.521554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.521829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.522523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.524423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.543670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:05:15.553899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1159/2347] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:05:15 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:05:15.554792 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.565324 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc705b-db6a-478f-a327-c40135503586 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.620807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.621409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.621712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.621995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.622675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.624638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.644397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1189/2348] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:05:15 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:05:15.648445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.658677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9082dbbb-7663-4b85-8464-2fad84a8cb27 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.663312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.663974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.664265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.664549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.665277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.666864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.677623 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1160/2349] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:05:15.681412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.691847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8ce19b-9636-4693-8e40-e245ec45e921 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.696802 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002423cf-e3ae-47b3-a361-bde35522aaba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.697453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002423cf-e3ae-47b3-a361-bde35522aaba admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.697737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002423cf-e3ae-47b3-a361-bde35522aaba admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.698014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002423cf-e3ae-47b3-a361-bde35522aaba admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.698731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002423cf-e3ae-47b3-a361-bde35522aaba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.700404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002423cf-e3ae-47b3-a361-bde35522aaba admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.710384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1190/2350] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:05:15.714357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.724901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f688be0-c1b5-4f53-8cea-4a6857b2f2fb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.729436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.730297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.730578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.730850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] PATH_INFO: `/v3/projects/7b3bdaad79fa4b2e9fc4444b8c62542f/users/f6e9f478c6e1497e843fd1aef1abc2fc/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.746460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3bdaad79fa4b2e9fc4444b8c62542f, user_id=f6e9f478c6e1497e843fd1aef1abc2fc, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.749116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.761366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1161/2351] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:05:15 2022] PUT /identity/v3/projects/7b3bdaad79fa4b2e9fc4444b8c62542f/users/f6e9f478c6e1497e843fd1aef1abc2fc/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:05:15.771509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb6313-18ad-4098-9b04-7ef837a6f407 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.771811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb6313-18ad-4098-9b04-7ef837a6f407 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.772101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb6313-18ad-4098-9b04-7ef837a6f407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.798825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ddfb6313-18ad-4098-9b04-7ef837a6f407 None None] MFA Rules not processed for user `f6e9f478c6e1497e843fd1aef1abc2fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:05:15.800071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ddfb6313-18ad-4098-9b04-7ef837a6f407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:15.863808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1191/2352] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:05:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:05:15.874917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:15.876605 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:15.955453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867e8dfb-8b2a-4b3a-b5f1-21687d9f85b7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:15.959193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:15.959871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:15.960159 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:15.960439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:15.969114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:15.970560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:15.979346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1162/2353] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:05:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:05:16.025572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:16.026000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:16.026343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:16.053995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:05:16.080182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:05:16.081633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:16.149196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1192/2354] 10.222.0.45 () {58 vars in 967 bytes} [Mon Jun 20 06:05:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:05:16.182161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:16.183771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:16.296644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeac4b7-99d3-4da2-8698-a9f024b5f3d2 service glance] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:16.302821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:16.303508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:16.303799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:16.304085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:16.313232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:16.315616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:16.326710 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1163/2355] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:05:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:05:20.713505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:05:20.715415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:20.776646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f64318-f9a6-46b8-86dd-9af0e6b9f740 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:05:20.783845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:05:20.784305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:05:20.784587 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:05:20.784862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:05:20.786758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:05:20.869074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:05:20.870666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:05:20.884936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1193/2356] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:05:20 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:05:26.321158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:05:30.876546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=82 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:06:27.005916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:06:27.007482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:27.019701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea6e7e3-b7bc-458f-b8c7-7672a2b697e3 service swift] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:06:27.023238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:06:27.023899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:06:27.024184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:06:27.024459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:06:27.026133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:27.109368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:06:27.110808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:06:27.130815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1164/2357] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:06:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:06:37.128205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=43 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:06:41.359970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:06:41.361368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:41.419940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb2e88f-8943-42cb-9586-b45330d2e631 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:06:41.423580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:06:41.424234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:06:41.424520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:06:41.424799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:06:41.426579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:41.484645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:06:41.486085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:06:41.494948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1194/2358] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:06:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:06:45.743652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:06:45.745289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:45.759989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b145d716-083a-43b3-b7c4-4f9abf516319 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:06:45.766223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:06:45.766910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:06:45.767204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:06:45.767514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:06:45.769359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:45.828091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:06:45.829686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:06:45.838784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1165/2359] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:06:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:06:47.144479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:06:47.153183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e5990c-8190-4292-bba8-f6d38b3777c2 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:06:47.156968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:06:47.157615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:06:47.157901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:06:47.158181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:06:47.159967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:06:47.221027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:06:47.222522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:06:47.231765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1195/2360] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:06:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:06:55.834507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=28 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:06:57.227472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=76 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:07:17.318648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:07:17.320146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:07:17.350085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9971d-b9a9-4523-bb82-08e2aa282d3e service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:07:17.371436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:07:17.376240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:07:17.376995 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:07:17.377957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:07:17.384173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:07:17.406120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:07:17.408035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:07:17.436553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1166/2361] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:07:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:07:17.520156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:07:17.532652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073834c0-5727-4b36-8a7b-598eee13c040 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:07:17.537365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:07:17.538093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:07:17.539320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:07:17.539715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:07:17.541859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:07:17.555209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:07:17.557013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:07:17.568311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1196/2362] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:07:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:07:18.839955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:07:18.849912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d8b722-a2d6-42ca-93f7-12b97a7be6f7 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:07:18.854191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:07:18.854896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:07:18.855435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:07:18.855722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:07:18.857563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:07:18.872425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:07:18.873840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:07:18.884941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1167/2363] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:07:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:07:27.563520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:07:28.880394 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=12 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:08:09.342556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:08:09.351720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa4852-f643-4803-bc6c-6b92f795f539 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:08:09.355293 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:08:09.355926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:08:09.356214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:08:09.356495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:08:09.358245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:08:09.416272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:08:09.417776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:08:09.429113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1197/2364] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:08:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:08:19.423197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:09:13.468815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:09:13.479839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6507bd-2af4-40e2-a757-033f94ec8bc9 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:09:13.485911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:09:13.486542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:09:13.486827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:09:13.487104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:09:13.488889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:09:13.552625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:09:13.554201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:09:13.563182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1168/2365] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:09:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:09:23.558967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=28 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:10:16.103202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:10:16.105128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:10:16.115902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09f1645-3af0-4428-895a-a23159f6596d service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:10:16.119593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:10:16.120346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:10:16.121244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:10:16.121600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:10:16.130016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:10:16.131476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:10:16.140288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1198/2366] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:10:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:10:26.136713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:10:42.741962 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:10:42.743409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:10:42.811064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577190-3255-4ddd-b022-fd5bd007233a service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:10:42.816119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:10:42.816780 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:10:42.817070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:10:42.817353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:10:42.819151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:10:42.832712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:10:42.834300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:10:42.845206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1169/2367] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:10:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:10:52.840875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=32 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:11:27.108598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:11:27.117126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3783f0-9fd7-4a06-acb9-e344551e9d08 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:11:27.120697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:11:27.121328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:11:27.121611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:11:27.121885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:11:27.131870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:11:27.133315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:11:27.143944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1199/2368] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:11:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:11:37.139192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:11:41.226526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:11:41.235883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05f0e51-da55-41a3-96f0-88523ab18f3a service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:11:41.239556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:11:41.240189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:11:41.240502 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:11:41.240781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:11:41.249143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:11:41.251015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:11:41.277794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1170/2369] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:11:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:11:46.546545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:11:46.557899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f30feb-c4ae-46b2-a8fd-e23ff47f230b service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:11:46.563445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:11:46.564107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:11:46.564392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:11:46.564672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:11:46.573216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:11:46.574825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:11:46.601871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1200/2370] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:11:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:11:47.136395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:11:47.149226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc30e67-a9e9-4519-bbc0-897c080a5f3c service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:11:47.154622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:11:47.155273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:11:47.155579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:11:47.155859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:11:47.164310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:11:47.165889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:11:47.193055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1171/2371] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:11:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:11:56.599316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=13 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:11:57.190353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=26 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:12:19.596857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:12:19.606596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5202c-d204-422b-81c0-b3f576e0e2bc service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:12:19.610177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:12:19.610809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:12:19.611095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:12:19.611422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:12:19.621986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:12:19.623445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:12:19.634022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1201/2372] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:12:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:12:29.629328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:12:42.425786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:12:42.435540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90bd94-b2a1-446f-90c3-3f97770309a2 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:12:42.439295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:12:42.439945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:12:42.440227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:12:42.440501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:12:42.448654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:12:42.450140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:12:42.460040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1172/2373] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:12:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:12:52.456379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:13:09.358713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:13:09.367949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbfd6b0-e033-465b-9d55-deeb0cd7cc74 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:13:09.371580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:13:09.372210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:13:09.372491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:13:09.372765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:13:09.380926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:13:09.382367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:13:09.391414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1202/2374] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:13:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:13:19.387141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:14:13.197631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:14:13.210559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b41b8-26fc-459a-b4b6-af1505722c8c service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:14:13.214440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:14:13.215072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:14:13.215400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:14:13.215681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:14:13.237430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:14:13.238849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:14:13.265566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1173/2375] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:14:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:14:23.262974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=21 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:15:15.709654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:15:15.746951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16496d10-8b84-4c89-b106-e0515edfc5fd service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:15:15.750570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:15:15.751206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:15:15.751520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:15:15.751798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:15:15.760122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:15:15.761552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:15:15.770311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1203/2376] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:15:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:15:25.766272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=18 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:15:43.792361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:15:43.793774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:15:43.858202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4b182-1123-42b4-8b1e-dd35a9eeff1b service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:15:43.863606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:15:43.864291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:15:43.864579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:15:43.864863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:15:43.873350 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:15:43.874923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:15:43.884785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1174/2377] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:15:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:15:53.880558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=25 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:16:26.806607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:16:26.872918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33b3e0-e54c-4c33-82fa-472a46b090d3 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:16:26.876564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:16:26.877276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:16:26.877479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:16:26.877758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:16:26.894793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:16:26.896262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:16:26.907044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1204/2378] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:16:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:16:36.902230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=33 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:16:40.813032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:16:40.891202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e4e4b-e8f4-4e6b-be21-b986833b11ac service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:16:40.896630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:16:40.897760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:16:40.898041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:16:40.898315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:16:40.906087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:16:40.907681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:16:40.917037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1175/2379] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:16:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:16:45.857396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:16:45.858983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:16:45.926421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edd6deb-39b5-436a-a2b7-3e9c802449c4 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:16:45.930018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:16:45.930665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:16:45.930951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:16:45.931229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:16:45.933142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:16:46.009435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:16:46.010872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:16:46.019784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1205/2380] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:16:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:16:47.501315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:16:47.512930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10162ef2-dd8d-48eb-8f96-cd304bb4b575 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:16:47.518541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512efbd-e958-476f-a983-faec09414285 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:16:47.519218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512efbd-e958-476f-a983-faec09414285 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:16:47.519538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512efbd-e958-476f-a983-faec09414285 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:16:47.519823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512efbd-e958-476f-a983-faec09414285 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:16:47.523155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7512efbd-e958-476f-a983-faec09414285 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:16:47.595556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512efbd-e958-476f-a983-faec09414285 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:16:47.597158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7512efbd-e958-476f-a983-faec09414285 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:16:47.606564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1176/2381] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:16:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:16:56.015589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=58 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:16:57.602205 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=71 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:17:16.901929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:17:16.903518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:17:16.997531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064813bd-6c85-4f01-aff7-1478f9f95b0f service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:17:17.002089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:17:17.002956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:17:17.003237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:17:17.003535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] PATH_INFO: `/v3/users/f6e9f478c6e1497e843fd1aef1abc2fc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:17:17.017195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e9f478c6e1497e843fd1aef1abc2fc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:17:17.019111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:17:17.064527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e9f478c6e1497e843fd1aef1abc2fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.071329 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e9f478c6e1497e843fd1aef1abc2fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.071984 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e9f478c6e1497e843fd1aef1abc2fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.072595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e9f478c6e1497e843fd1aef1abc2fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.084139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e9f478c6e1497e843fd1aef1abc2fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.097437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1206/2382] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:17:16 2022] DELETE /identity/v3/users/f6e9f478c6e1497e843fd1aef1abc2fc => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:17:17.101922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:17:17.102994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:17:17.103105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:17:17.130083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:17:17.131364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:17:17.190079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1177/2383] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:17:17.200185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:17:17.201624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:17:17.248340 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:17:17.275449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0057a-865b-4531-8bbb-5b4f65f94c03 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:17:17.279134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:17:17.279819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:17:17.280109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:17:17.280393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:17:17.289285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:17:17.291312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:17:17.300664 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1207/2384] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:17:17.562336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:17:17.563783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:17:17.578388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d394c495-e084-4d7c-b832-994411fce8a2 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:17:17.584221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:17:17.585098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:17:17.585384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:17:17.585656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] PATH_INFO: `/v3/projects/7b3bdaad79fa4b2e9fc4444b8c62542f` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:17:17.601333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3bdaad79fa4b2e9fc4444b8c62542f)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:17:17.603233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:17:17.659829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3bdaad79fa4b2e9fc4444b8c62542f', 'request_id': 'req-b8678759-dc94-45c4-bba9-62ce0de3eef6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.665902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3bdaad79fa4b2e9fc4444b8c62542f', 'request_id': 'req-b8678759-dc94-45c4-bba9-62ce0de3eef6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.672658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3bdaad79fa4b2e9fc4444b8c62542f', 'request_id': 'req-b8678759-dc94-45c4-bba9-62ce0de3eef6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.685557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] The token cache is being invalidate because project 7b3bdaad79fa4b2e9fc4444b8c62542f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:17:17.685878 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.686504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:17:17.688240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1178/2385] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:17:17 2022] DELETE /identity/v3/projects/7b3bdaad79fa4b2e9fc4444b8c62542f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:17:21.805436 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:17:21.813801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a96b6-954c-4f63-a92d-2fc591c89ac3 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:17:21.869894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:17:21.870545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:17:21.870830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:17:21.871108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:17:21.872909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:17:21.953267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:17:21.954890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:17:21.966189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1208/2386] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:17:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:17:27.682607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=61,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:17:31.960624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=72 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:18:09.255371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:18:09.257162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:18:09.273009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8678759-dc94-45c4-bba9-62ce0de3eef6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:18:09.278520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:18:09.279177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:18:09.279497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:18:09.279783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:18:09.281549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:18:09.364282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:18:09.365810 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:18:09.384264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1179/2387] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:18:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:18:19.302310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:18:19.303843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:18:19.318252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c633f7-adf1-42c4-8e07-a208053594a5 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:18:19.324233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:18:19.324884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:18:19.325174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:18:19.325456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:18:19.328188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:18:19.344707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:18:19.346709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:18:19.358004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1209/2388] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:18:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:18:19.381541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=41 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:18:19.452210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:18:19.453938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:18:19.466519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd6a05-53c8-4c38-8b29-fabcce32a7b8 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:18:19.470586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:18:19.471525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:18:19.471813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:18:19.472097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:18:19.473919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:18:19.484318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:18:19.485748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:18:19.494051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1180/2389] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:18:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:18:29.353073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:18:29.490236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:19:13.373087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:19:13.374481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:19:13.431226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07aacd85-b9c6-4eff-be2b-9c7ab5b9f1b8 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:19:13.434875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:19:13.435585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:19:13.435871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:19:13.436150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:19:13.438583 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:19:13.510313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:19:13.511794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:19:13.523393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1210/2390] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:19:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:19:23.519113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=53 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:20:48.781509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:20:48.782946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:20:48.863481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f786e23-319c-4159-92d4-73b34fe24442 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:20:48.868479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:20:48.869227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:20:48.869517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:20:48.869796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:20:48.871579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:20:48.883832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:20:48.885391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:20:48.895114 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1181/2391] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:20:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:20:58.890980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=32 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:21:26.898184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:21:26.907441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a23cca-8dd8-41ae-b224-ffe9165744c9 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:21:26.911139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:21:26.911892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:21:26.912242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:21:26.912585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:21:26.914395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:21:26.928314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:21:26.930270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:21:26.941650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1211/2392] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:21:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:21:36.936504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:21:41.395593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:21:41.397056 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:21:41.408343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b93cc-3d6b-4747-b0b7-00f86c31702e service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:21:41.411973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:21:41.412614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:21:41.412902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:21:41.413182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:21:41.415455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:21:41.496932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:21:41.498555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:21:41.517089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1182/2393] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:21:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:21:47.674242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:21:47.683953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6eb84b-fa12-4410-99a8-19640632eca2 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:21:47.687546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:21:47.688174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:21:47.688458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:21:47.688736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:21:47.690506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:21:47.757494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:21:47.759531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:21:47.777206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1212/2394] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:21:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:21:51.514501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=41 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:21:57.775220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=36 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:22:17.266763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:22:17.275220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16fedb8-fa19-4ea7-b1d1-56ae6245117b service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:22:17.280719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:22:17.282413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:22:17.283484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:22:17.283951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:22:17.286084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:22:17.358333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:22:17.359811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:22:17.378594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1183/2395] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:22:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:22:22.965318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:22:22.966726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:22:22.978468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18297919-d300-4cfe-9908-cefc3bbbf5a4 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:22:22.982035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:22:22.982670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:22:22.982955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:22:22.983231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:22:22.993430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:22:22.994851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:22:23.006010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1213/2396] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:22:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:22:27.375846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=36 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:22:33.001253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:23:09.448990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:23:09.462335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30227641-fc24-4a89-b13b-e4f063d5314c service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:23:09.468027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:23:09.468674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:23:09.468960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:23:09.469236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:23:09.477421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:23:09.478983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:23:09.488275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1184/2397] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:23:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:23:19.483647 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:23:23.330820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:23:23.342175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1e07-8c5c-4bde-9981-a7260b614f90 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:23:23.348682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:23:23.349343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:23:23.349634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:23:23.349917 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:23:23.351999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:23:23.365951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:23:23.367752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e8c376-bc77-4e27-acdb-a8c248b243be service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:23:23.379439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1214/2398] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:23:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:23:33.374482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:23:54.456830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:23:54.466132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf33cec-7bae-4701-b7cc-c596d042887c service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:23:54.470300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:23:54.470998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:23:54.471280 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:23:54.471568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:23:54.479551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:23:54.480982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:23:54.490642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1185/2399] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:23:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:24:03.150477 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:24:03.151004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:24:03.151128 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:24:03.177724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] MFA Rules not processed for user `70323e3276ca4aaa87f55c5fb38837aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:24:03.178925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:24:03.186717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1215/2400] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:24:03 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:24:03.485750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:24:03.494483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af08cbaa-196b-4a7f-8b2d-d3f2e2b8c584 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:24:03.498284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:24:03.498926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:24:03.499217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:24:03.499530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:24:03.501346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:24:03.510754 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:24:03.512213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:24:03.520156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1186/2401] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:24:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:24:03.842355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:24:03.854944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff79b5b-3304-4555-a132-53e83f926990 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:24:03.860539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:24:03.861430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:24:03.861743 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:24:03.862033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:24:03.863864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:24:03.876866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:24:03.878465 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:24:03.888993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1216/2402] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:24:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:24:10.297019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:24:10.310712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc7183c-aa0e-4052-8024-6b2e3f0a6f35 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:24:10.316711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:24:10.317357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:24:10.317643 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:24:10.317921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:24:10.319690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:24:10.332495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:24:10.333992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:24:10.344925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1187/2403] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:24:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:24:13.321630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:24:13.347612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf892db6-0c5d-46bd-bf67-0437bbadf59c service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:24:13.353353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c025f-1224-4806-864b-1ed6f4d0e753 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:24:13.353996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c025f-1224-4806-864b-1ed6f4d0e753 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:24:13.354281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c025f-1224-4806-864b-1ed6f4d0e753 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:24:13.354556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c025f-1224-4806-864b-1ed6f4d0e753 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:24:13.363624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c025f-1224-4806-864b-1ed6f4d0e753 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:24:13.365207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c025f-1224-4806-864b-1ed6f4d0e753 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:24:13.376501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1217/2404] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:24:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:24:14.592572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:24:14.618068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ed880-e438-4663-b0a0-5f0fc882ccaa service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:24:14.623626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:24:14.624272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:24:14.624564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:24:14.624847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:24:14.632632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:24:14.634209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:24:14.662304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1188/2405] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:24:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:24:23.371309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=24 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:24:24.659572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=37 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:21.110034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:21.110514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:21.110623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:21.167156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] MFA Rules not processed for user `306d74f4500845a097e8e062af12dc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:21.168445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:21.243628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1218/2406] 10.222.0.45 () {58 vars in 967 bytes} [Mon Jun 20 06:25:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:21.287984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:21.289392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:21.382038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2c677-1138-49b2-9308-c1298346a8c5 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:21.387041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b3eae-d66d-4791-aa91-3c58a371fa04 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a7ac2b5986a74ea9923fa0593e58a6b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:21.387716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b3eae-d66d-4791-aa91-3c58a371fa04 service swift] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:21.388000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b3eae-d66d-4791-aa91-3c58a371fa04 service swift] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:21.388274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b3eae-d66d-4791-aa91-3c58a371fa04 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:21.396189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b3eae-d66d-4791-aa91-3c58a371fa04 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:21.397736 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b3eae-d66d-4791-aa91-3c58a371fa04 service swift] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:21.405565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1189/2407] 10.222.0.45 () {60 vars in 1384 bytes} [Mon Jun 20 06:25:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 20 06:25:21.925906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:21.935592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3504f8e-5877-4089-81c1-eb3e68ec3c4f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:21.939147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:21.939859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:21.940148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:21.940431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:21.942290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:22.005367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:22.006727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8173f-b149-4399-b150-8d583c4576ce service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:22.016240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1219/2408] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:25:24.601136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.601650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.601792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.651884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:24.653194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:24.661740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1190/2409] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:24.669635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.670127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.670234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.687777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:24.688985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:24.696828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1220/2410] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:25:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:24.700746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:24.710790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63695c01-ca29-43c1-9ad5-b93d53519e92 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:24.715343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:24.716004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.716279 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.716559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.717287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:25:24.717595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:24.719173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:24.731905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1191/2411] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:25:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:25:24.736008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:24.745246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc46f1-a640-46b0-9f7a-b99b76ce7e6d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:24.749880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:24.750521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.750814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.751095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.751839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:24.753749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:24.790214 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1221/2412] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:25:24 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:25:24.794337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:24.804239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdd55a-6887-400c-97de-784bed60217a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:24.861596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:24.862267 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.862554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.862836 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.863560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:24.865467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:24.884964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1192/2413] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:25:24 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:25:24.888998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:24.899544 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a887fc3-f6ba-40b5-8c47-39a62ab1db0b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:24.904069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:24.904719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.905003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.905282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.905996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:24.907636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:24.908841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:24.922081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1222/2414] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:25:24.923204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:24.932888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f7cb2e-4c45-41ca-b618-9991abf8da0e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:24.937452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c6827-3cea-4d9e-9178-70607973fe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:24.938100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6827-3cea-4d9e-9178-70607973fe54 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.938390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6827-3cea-4d9e-9178-70607973fe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.938686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c6827-3cea-4d9e-9178-70607973fe54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.939438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c6827-3cea-4d9e-9178-70607973fe54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:24.941011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0c6827-3cea-4d9e-9178-70607973fe54 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:24.946045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:24.954421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1193/2415] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:25:24.955735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:24.966947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dc6bc-34de-49fe-9356-05321248e7b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:24.972121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:24.973486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:24.973769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:24.974042 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] PATH_INFO: `/v3/projects/aee5c8d850d143cbac59ea9fb688ac68/users/4c005728531546aabec0720d9bb1c636/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:24.990120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee5c8d850d143cbac59ea9fb688ac68, user_id=4c005728531546aabec0720d9bb1c636, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:24.992892 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:25.007380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1223/2416] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:25:24 2022] PUT /identity/v3/projects/aee5c8d850d143cbac59ea9fb688ac68/users/4c005728531546aabec0720d9bb1c636/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:25:25.012464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401922d7-5464-479a-9355-2493bbc4e4f8 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:25.012758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401922d7-5464-479a-9355-2493bbc4e4f8 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:25.013039 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401922d7-5464-479a-9355-2493bbc4e4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:25.030872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-401922d7-5464-479a-9355-2493bbc4e4f8 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:25.032080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-401922d7-5464-479a-9355-2493bbc4e4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:25.091584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1194/2417] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:25.108012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:25.163363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4947e2-dada-45c5-87de-1ee1a6f95747 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:25.167015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:25.167715 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:25.168126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:25.168412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:25.178011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:25.179489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:25.189749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1224/2418] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:25:29.273798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.274368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.274368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.310742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] MFA Rules not processed for user `4c005728531546aabec0720d9bb1c636`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:29.312022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:29.382268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1195/2419] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:25:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:29.387096 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:29.397551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea22c5f-938f-475f-ab62-c26850fb4a76 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:29.402456 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:29.403132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.403516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.403860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.404624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:29.406612 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:29.435305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1225/2420] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:25:29 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:25:29.439339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:29.450336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3ff120-d1f1-444f-876d-586d43f38bad None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:29.512898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:29.513580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.513872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.514156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.514864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:29.516829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:29.538162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1196/2421] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:25:29 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:25:29.541815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:29.555595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c3757-73d4-46c9-9dca-9f78369b0962 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:29.560396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:29.561152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.561512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.561866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.562608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:29.564278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:29.577149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1226/2422] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:25:29.581390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:29.592483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4b7b3-2f0e-4d52-8f86-c8def187b585 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:29.597133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:29.598571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.598855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.599132 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] PATH_INFO: `/v3/projects/8ce83c74885b429eb85d9bbd65e53c5f/users/8a98b2c69e1e49dfa7858b8616a538d8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.614887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce83c74885b429eb85d9bbd65e53c5f, user_id=8a98b2c69e1e49dfa7858b8616a538d8, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:29.617550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:29.620561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:29.631829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1197/2423] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:25:29 2022] PUT /identity/v3/projects/8ce83c74885b429eb85d9bbd65e53c5f/users/8a98b2c69e1e49dfa7858b8616a538d8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:25:29.635706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:29.646380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb6efd6-ba8b-457a-b3b5-af95c495cd55 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:29.701170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3e64-4825-4c53-86b1-bfde93917e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:29.701832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3e64-4825-4c53-86b1-bfde93917e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.702138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3e64-4825-4c53-86b1-bfde93917e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.702419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3e64-4825-4c53-86b1-bfde93917e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.703134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3e64-4825-4c53-86b1-bfde93917e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:29.704774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3e64-4825-4c53-86b1-bfde93917e48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:29.714844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1227/2424] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:25:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:25:29.719071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:29.729523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b9d36-8d92-463f-ba7e-770d184de5ff admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:29.734215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:29.735098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:29.735416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:29.735696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] PATH_INFO: `/v3/projects/8ce83c74885b429eb85d9bbd65e53c5f/users/8a98b2c69e1e49dfa7858b8616a538d8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:29.737670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce83c74885b429eb85d9bbd65e53c5f, user_id=8a98b2c69e1e49dfa7858b8616a538d8, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:29.740361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:29.756759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1198/2425] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:25:29 2022] PUT /identity/v3/projects/8ce83c74885b429eb85d9bbd65e53c5f/users/8a98b2c69e1e49dfa7858b8616a538d8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:25:34.433142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1be0-f33a-4f63-878d-bfc8ef4ea5f8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:34.433619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1be0-f33a-4f63-878d-bfc8ef4ea5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:34.433724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1be0-f33a-4f63-878d-bfc8ef4ea5f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:34.464813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-975f1be0-f33a-4f63-878d-bfc8ef4ea5f8 None None] MFA Rules not processed for user `8a98b2c69e1e49dfa7858b8616a538d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:34.466055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-975f1be0-f33a-4f63-878d-bfc8ef4ea5f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:34.501295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:34.559301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1228/2426] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:25:34 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:34.573805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:34.648551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdc47f-c376-4336-90aa-3967fff2eb3f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:34.654539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:34.655355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:34.655718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:34.656062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:34.726709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:34.728359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:34.737936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1199/2427] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:25:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:25:35.684451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:35.684947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:35.685170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:35.704727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] MFA Rules not processed for user `4c005728531546aabec0720d9bb1c636`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:25:35.705972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:35.713038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1229/2428] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:25:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:25:44.733606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=63,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:45.698922 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=20 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:25:54.763150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:25:54.764613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:25:54.874567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9835482d-7212-4c02-89ba-06a4465f9578 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:25:54.880145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:25:54.880786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:25:54.881070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:25:54.881346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:25:54.889606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:25:54.891163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:25:54.901417 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1200/2429] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:25:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:26:04.897324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=51 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:26:47.170990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:26:47.199771 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4617d53-b848-4966-af9b-fb20e2d0832f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:26:47.205111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:26:47.205740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:26:47.206023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:26:47.206301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:26:47.260569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:26:47.262142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:26:47.270867 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1230/2430] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:26:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:26:57.266773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=35 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:27:22.590720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:27:22.653996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea3373-1f77-44cd-9c27-1c330089bf1a service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:27:22.657633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:27:22.658276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:27:22.658570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:27:22.658850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:27:22.660755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:27:22.686713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:27:22.688172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:27:22.699025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1201/2431] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:27:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:27:23.765635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:27:23.834238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3b837-4da6-473e-a68b-7329bf361a49 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:27:23.837860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:27:23.838502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:27:23.838785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:27:23.839156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:27:23.894240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:27:23.895699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:27:23.904511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1231/2432] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:27:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:27:32.694203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=35 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:27:33.900475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=52 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:28:18.267357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:28:18.277318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc609d0-81e1-4d61-82e4-1de3c2632722 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:28:18.280967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:28:18.281618 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:28:18.281904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:28:18.282189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:28:18.341371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:28:18.342908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:28:18.353987 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1202/2433] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:28:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:28:18.466475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:28:18.475666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95168e-1cad-4096-99d7-d1c3170b106b service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:28:18.479501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:28:18.480133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:28:18.480418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:28:18.480700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:28:18.482501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:28:18.562300 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:28:18.563755 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:28:18.582080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1232/2434] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:28:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:28:28.348620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=28 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:28:28.579430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=41 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:29:02.444916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:02.447035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:29:02.471509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd13466-78a4-42bf-9599-1b89581f61db service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:02.475439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:02.476119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:02.476663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:02.476941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:02.478901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:29:02.489944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:02.491415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26804de0-1a25-40b2-9e64-aff573d46394 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:02.500891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1203/2435] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:12.496136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=11 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:29:24.163336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:24.181879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa3949-cc8d-49bd-afa1-8f92ae955b81 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:24.187840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:24.188798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:24.189088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:24.189504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:24.191732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:29:24.288506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:24.290571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:24.305172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1233/2436] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:25.012604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:25.013723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:25.014250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:25.060451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] MFA Rules not processed for user `adab7140fc804689af945257511cfded`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:29:25.062679 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:29:25.070884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1204/2437] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:29:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:29:25.406292 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:25.427146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8718a62a-60cb-4e7e-862c-e6ad89bba469 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:25.431454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:25.432213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:25.432571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:25.432952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:25.443320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:25.445983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:25.456988 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1234/2438] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:25.750357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:25.751967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:29:25.782390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7deacb-d2fa-4d2f-885d-1d7b4e70ef66 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:25.788288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:25.789841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:25.790813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:25.791297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:25.791903 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:25.805462 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:25.805992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fe5ae8-d89a-41fb-81bc-896b2a3fd7b1 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:25.807774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:25.813799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:25.825858 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:25.827773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:25.829060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:25.832352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1205/2439] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:25.844797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:25.846429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:25.874066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1235/2440] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:34.128233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:34.141792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce425a7c-de4e-403f-8000-d886d982c1ae service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:34.148565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:34.149329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:34.149676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:34.150031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:34.160635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:34.162233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:34.174536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1206/2441] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:35.853742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=45 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:29:37.526425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:29:37.540473 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c108bc29-28dd-46a3-8870-d553b3be78b9 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:29:37.547374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:29:37.548306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:29:37.548660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:29:37.549003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:29:37.551028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:29:37.567079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:29:37.569144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:29:37.582726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1236/2442] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:29:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:29:44.169371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=23 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:29:47.576669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:30:56.749853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:30:56.751545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:30:56.767037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e03491-e5bf-47c1-95ac-7353c8f97be4 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:30:56.771988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:30:56.772628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:30:56.772913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:30:56.773189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:30:56.775016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:30:56.786654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:30:56.788330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:30:56.798220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1207/2443] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:30:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:31:06.793981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:31:24.581740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:31:24.592531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55252a0d-2307-46b2-916a-bf85a51af77e service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:31:24.596740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:31:24.597496 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:31:24.597879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:31:24.598254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:31:24.608475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:31:24.610186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:31:24.623039 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1237/2444] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:31:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:31:34.616262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:32:19.110294 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:19.111953 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:19.137395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbb0c5-6e52-41d4-92d9-1e96b2009463 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:19.140981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:19.141633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:19.141917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:19.142194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:19.144076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:19.166671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:19.168759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:19.190078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1208/2445] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:32:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:32:23.560405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:23.562107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:23.577688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e74ab-bffa-4f84-8ec6-d07eeef44a67 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:23.583704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:23.585467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:23.585995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:23.586503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:23.610899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:23.612873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:23.639607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1238/2446] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:32:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:32:24.743896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:24.753904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8ec048-da13-461e-bf93-e93e99166e7e service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:24.757537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:24.758186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:24.758481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:24.758758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:24.760708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:24.858629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:24.860083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:24.879917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1209/2447] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:32:27.726705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:27.747003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4732f204-a447-4743-9a9b-ec11f0a6b28d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:27.753059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:27.754375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:27.755004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:27.755518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:27.757762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:27.782026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:27.783956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:27.798971 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1239/2448] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:32:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:32:27.805015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:27.807127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:27.841881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ea930b-7eee-4787-8ff5-a9af56e040b7 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:27.846720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:27.848282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:27.848773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:27.849315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] PATH_INFO: `/v3/users/ca882ae475a24e7f84124a1ab3e374f6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:27.867844 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca882ae475a24e7f84124a1ab3e374f6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:27.870384 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:27.930663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca882ae475a24e7f84124a1ab3e374f6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:27.937523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca882ae475a24e7f84124a1ab3e374f6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:27.938194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca882ae475a24e7f84124a1ab3e374f6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:27.938820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca882ae475a24e7f84124a1ab3e374f6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:27.947628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:32:27.954186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-25aa58f7-d45c-4eb4-9fea-cd8532f1fe83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca882ae475a24e7f84124a1ab3e374f6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:27.969049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1210/2449] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:32:27 2022] DELETE /identity/v3/users/ca882ae475a24e7f84124a1ab3e374f6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:32:28.156369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.167742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff2712-05a5-4a86-ba78-f6922f94ca2e service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.229671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.230533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.230815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.231090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] PATH_INFO: `/v3/projects/e339d5680b35466b8df59e6770716c4f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.243762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e339d5680b35466b8df59e6770716c4f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.246336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.296983 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e339d5680b35466b8df59e6770716c4f', 'request_id': 'req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:28.302536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e339d5680b35466b8df59e6770716c4f', 'request_id': 'req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:28.310322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e339d5680b35466b8df59e6770716c4f', 'request_id': 'req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:28.322508 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] The token cache is being invalidate because project e339d5680b35466b8df59e6770716c4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:32:28.322798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:28.324145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0babedfe-1270-4ada-9bb3-11b7e6f0be6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:32:28.325751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1240/2450] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:32:28 2022] DELETE /identity/v3/projects/e339d5680b35466b8df59e6770716c4f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:32:28.331892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.332189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.332472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.361682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:32:28.362919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:28.439482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1211/2451] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:32:28.447993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.448966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.449377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.468144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:32:28.469482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:28.478155 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1241/2452] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:32:28.483004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.493449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c880c5-539b-444e-a2c2-f6cad8fe734c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.500130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.500809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.501104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.501386 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.502213 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:32:28.502654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.504368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.518102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1212/2453] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:32:28.522378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.532589 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c9a1d-b444-4a0f-a604-f6bf1fe0ddf6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.538565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.539243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.539563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.539841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.540541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.543056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.573132 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1242/2454] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:32:28 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:32:28.577510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.588221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d246053-45d0-4a87-a905-d925466ba194 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.654232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.654917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.655200 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.655693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.656774 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.658797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.680542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1213/2455] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:32:28 2022] POST /identity/v3/users => generated 302 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:32:28.685390 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.697274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fe4de9-bd2b-41bb-ba56-f5587d0569d1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.702655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.731934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.731934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.731934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.731934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.731934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.731934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1243/2456] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:32:28.739758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.745905 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46c43b-460a-44c2-ad1a-71fbf5fd8136 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.750985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c2df07-6da4-43bf-b999-8754d9b865ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.751748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2df07-6da4-43bf-b999-8754d9b865ef admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.752038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2df07-6da4-43bf-b999-8754d9b865ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.752322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c2df07-6da4-43bf-b999-8754d9b865ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.753067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2df07-6da4-43bf-b999-8754d9b865ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.754718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c2df07-6da4-43bf-b999-8754d9b865ef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.766257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1214/2457] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:32:28.770647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:28.781088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afce0d0-3a84-4a88-8365-9f5d40225788 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:28.785738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:28.786625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.786918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.787200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] PATH_INFO: `/v3/projects/b890efe4f38642aba2196a7d4dd35871/users/5d3f1ae613b642eab6aa1257085dc8b7/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.800232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:32:28.804499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b890efe4f38642aba2196a7d4dd35871, user_id=5d3f1ae613b642eab6aa1257085dc8b7, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:28.808722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:28.829662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1244/2458] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:32:28 2022] PUT /identity/v3/projects/b890efe4f38642aba2196a7d4dd35871/users/5d3f1ae613b642eab6aa1257085dc8b7/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:32:28.840890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:28.841377 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:28.841474 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:28.868806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] MFA Rules not processed for user `5d3f1ae613b642eab6aa1257085dc8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:32:28.870036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:28.902757 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:32:28.938392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1215/2459] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:32:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:32:28.943045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:29.006485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d0a5a2-78f4-4070-a766-cb8e3400fa95 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:29.011079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:29.011769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.012057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.012338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.013041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:29.014952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:29.040540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1245/2460] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:32:28 2022] POST /identity/v3/projects => generated 403 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:32:29.045346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:29.056901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da3ccf-ad9c-43b0-a3b3-986de2554490 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:29.115435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:29.116116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.116407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.116705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.117403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:29.119344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:29.139768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1216/2461] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:32:29 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:32:29.144011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:29.154810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbea79-323e-4889-a9c9-8eb28f79eaf3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:29.159514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:29.160176 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.160467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.160749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.161469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:29.163277 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:29.174380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1246/2462] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:32:29.178768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:29.188916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3ed31-61e8-4c65-89f5-85b0a0f1776d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:29.193782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:29.195125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.195125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.195373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] PATH_INFO: `/v3/projects/459a5416180b407c9b0ee1b3a53f56bc/users/01f1b7c326aa42029d500407cac87f2a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.212288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459a5416180b407c9b0ee1b3a53f56bc, user_id=01f1b7c326aa42029d500407cac87f2a, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:29.215758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:29.230574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1217/2463] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:32:29 2022] PUT /identity/v3/projects/459a5416180b407c9b0ee1b3a53f56bc/users/01f1b7c326aa42029d500407cac87f2a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:32:29.234720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:29.244783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3c5f53-e3db-44cf-8255-306d1f497e6b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:29.302908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f59ea9-0d84-442b-a3f3-b7c117f7e711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:29.303537 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59ea9-0d84-442b-a3f3-b7c117f7e711 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.303835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59ea9-0d84-442b-a3f3-b7c117f7e711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.304135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f59ea9-0d84-442b-a3f3-b7c117f7e711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.304838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f59ea9-0d84-442b-a3f3-b7c117f7e711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:29.306452 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f59ea9-0d84-442b-a3f3-b7c117f7e711 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:29.316519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1247/2464] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:32:29.320866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:32:29.331855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2d87b-6f10-42ab-808d-c770ce3a8924 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:32:29.336609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:32:29.337495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.337779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.338057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] PATH_INFO: `/v3/projects/459a5416180b407c9b0ee1b3a53f56bc/users/01f1b7c326aa42029d500407cac87f2a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.340604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=459a5416180b407c9b0ee1b3a53f56bc, user_id=01f1b7c326aa42029d500407cac87f2a, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:32:29.343223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:32:29.353735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1218/2465] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:32:29 2022] PUT /identity/v3/projects/459a5416180b407c9b0ee1b3a53f56bc/users/01f1b7c326aa42029d500407cac87f2a/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:32:29.365270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:32:29.365568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:32:29.365852 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:32:29.392787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] MFA Rules not processed for user `01f1b7c326aa42029d500407cac87f2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:32:29.394059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:32:29.442348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:32:29.466243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1248/2466] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:32:29 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:32:39.349785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=56,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:32:39.463726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:33:24.784888 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:33:24.786989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:33:24.852522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999146-1e09-40d3-96c1-0a17d01d8d26 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:33:24.856163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:33:24.856814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:33:24.857104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:33:24.857382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:33:24.859185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:33:24.939191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:33:24.940646 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:33:24.959820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1219/2467] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:33:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:33:34.957316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=61 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:34:19.436043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:34:19.438092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:34:19.451774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb432786-8715-4bbe-a551-cf260107c163 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:34:19.455816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:34:19.456553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:34:19.456921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:34:19.457269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:34:19.459207 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:34:19.548253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:34:19.549700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:34:19.559688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1249/2468] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:34:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:34:19.829481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:34:19.842225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d804602-db73-467a-a3d3-bc29fe0937e4 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:34:19.848548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:34:19.849876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:34:19.850408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:34:19.851403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:34:19.865982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:34:19.868550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:34:19.885491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1220/2469] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:34:19 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:34:29.554565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=34 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:34:29.878338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:35:25.100875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:35:25.113005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0569872-9ed1-4c0c-9f57-d6bc4ffdac44 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:35:25.116640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:35:25.117274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:35:25.117560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:35:25.117836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:35:25.120011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:35:25.188790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:35:25.190230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:35:25.207874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1250/2470] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:35:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:35:35.205282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=38 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:01.958742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:01.960197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:02.089289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b3cf37-5706-4d93-a4f7-046896739614 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:02.094510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:02.095165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:02.095484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:02.095766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:02.097554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:02.109512 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:02.111085 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:02.121784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1221/2471] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:12.118103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=57 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:19.345346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:19.347107 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:19.510033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2642c29-1805-4cce-b882-f61749ce8a63 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:19.515375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94fe016-7900-42ca-90bc-89d52e6bb765 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:19.515937 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fe016-7900-42ca-90bc-89d52e6bb765 service glance] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:19.516222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fe016-7900-42ca-90bc-89d52e6bb765 service glance] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:19.516556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94fe016-7900-42ca-90bc-89d52e6bb765 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:19.519209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d94fe016-7900-42ca-90bc-89d52e6bb765 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application [None req-d94fe016-7900-42ca-90bc-89d52e6bb765 service glance] Could not find project: 7b3bdaad79fa4b2e9fc4444b8c62542f.: keystone.exception.ProjectNotFound: Could not find project: 7b3bdaad79fa4b2e9fc4444b8c62542f. Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Jun 20 06:36:19.570877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:36:19.572199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self._validate_token_resources() Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 20 06:36:19.573317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.get_or_create( Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application with Lock( Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._enter() Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.creator() Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application created_value = creator( Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 20 06:36:19.574393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 20 06:36:19.575450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 7b3bdaad79fa4b2e9fc4444b8c62542f. Jun 20 06:36:19.575450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: ERROR keystone.server.flask.application Jun 20 06:36:19.577421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1251/2472] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:19 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) Jun 20 06:36:20.087157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:20.161080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d77125-2491-478c-8fec-d393f56f8ca4 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:20.166083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:20.166993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:20.167329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:20.167617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] PATH_INFO: `/v3/users/5d3f1ae613b642eab6aa1257085dc8b7` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:20.168818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d3f1ae613b642eab6aa1257085dc8b7)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:20.171841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:20.225050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d3f1ae613b642eab6aa1257085dc8b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:20.235082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d3f1ae613b642eab6aa1257085dc8b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:20.235855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d3f1ae613b642eab6aa1257085dc8b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:20.236469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d3f1ae613b642eab6aa1257085dc8b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:20.252930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d3f1ae613b642eab6aa1257085dc8b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:20.268565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1222/2473] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:20 2022] DELETE /identity/v3/users/5d3f1ae613b642eab6aa1257085dc8b7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:20.273605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:20.273968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:20.274368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:20.293109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:20.294339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:20.357426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1252/2474] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:20.370983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:20.372539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:20.444935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62320c35-2bce-45ff-8c51-8c4afae80aff admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:20.449932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:20.451001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:20.451423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:20.451805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:20.464694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:20.466510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:20.480525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1223/2475] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:36:20.842034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:20.843958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:20.870140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8a904-34c3-424b-8c81-268c15e97a92 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:20.900101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:20.901543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:20.901800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:20.902050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] PATH_INFO: `/v3/projects/b890efe4f38642aba2196a7d4dd35871` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:20.908276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b890efe4f38642aba2196a7d4dd35871)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:20.910273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:21.016546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b890efe4f38642aba2196a7d4dd35871', 'request_id': 'req-f44d6705-65f8-4d1c-905a-3a48d2a199bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.034167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b890efe4f38642aba2196a7d4dd35871', 'request_id': 'req-f44d6705-65f8-4d1c-905a-3a48d2a199bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.055997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b890efe4f38642aba2196a7d4dd35871', 'request_id': 'req-f44d6705-65f8-4d1c-905a-3a48d2a199bd'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.061154 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:21.076044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] The token cache is being invalidate because project b890efe4f38642aba2196a7d4dd35871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:21.076339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.077006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.078913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1253/2476] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:20 2022] DELETE /identity/v3/projects/b890efe4f38642aba2196a7d4dd35871 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:21.083018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:21.092839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af9b47e-c3fe-4630-b15b-d6db6e0596f9 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:21.120234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:21.172613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:21.173547 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:21.173934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:21.174123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] PATH_INFO: `/v3/users/01f1b7c326aa42029d500407cac87f2a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:21.175357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f1b7c326aa42029d500407cac87f2a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:21.177345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:21.230427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f1b7c326aa42029d500407cac87f2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.238508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f1b7c326aa42029d500407cac87f2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.239219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f1b7c326aa42029d500407cac87f2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.239856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f1b7c326aa42029d500407cac87f2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.254585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f1b7c326aa42029d500407cac87f2a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.272538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1224/2477] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:21 2022] DELETE /identity/v3/users/01f1b7c326aa42029d500407cac87f2a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:21.382460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:21.387074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:21.476500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d6705-65f8-4d1c-905a-3a48d2a199bd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:21.556388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:21.557175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:21.559599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:21.559599 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:21.560748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:21.574551 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:21.576339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:21.589061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1254/2478] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:21.680396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:21.682064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:21.840880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:21.845269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:21.877540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce59d0f-1c1f-4002-9037-419fb38800dd admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:21.887018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:21.888299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:21.888727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:21.889108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] PATH_INFO: `/v3/projects/459a5416180b407c9b0ee1b3a53f56bc` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:21.891037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=459a5416180b407c9b0ee1b3a53f56bc)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:21.893782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:21.931007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8504a3-0e59-4658-98f7-f1f733797dfc service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:21.935054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:21.935977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:21.936318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:21.936742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:21.938887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:21.949388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:21.955142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '459a5416180b407c9b0ee1b3a53f56bc', 'request_id': 'req-da1b6a5b-3394-47ae-acfa-e0856f3479d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.956011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:21.957485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde2936-1035-486c-b408-0246a33f43ed service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:21.966074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '459a5416180b407c9b0ee1b3a53f56bc', 'request_id': 'req-da1b6a5b-3394-47ae-acfa-e0856f3479d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.970102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1255/2479] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:36:21.972964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '459a5416180b407c9b0ee1b3a53f56bc', 'request_id': 'req-da1b6a5b-3394-47ae-acfa-e0856f3479d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.988783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] The token cache is being invalidate because project 459a5416180b407c9b0ee1b3a53f56bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:21.989193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.989921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-da1b6a5b-3394-47ae-acfa-e0856f3479d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:21.993098 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1225/2480] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:21 2022] DELETE /identity/v3/projects/459a5416180b407c9b0ee1b3a53f56bc => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:21.999441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:21.999933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.000353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.035369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:22.058617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:22.060008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:22.128012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1256/2481] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:36:22.136251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.137328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.137671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.162573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:22.163828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:22.175304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1226/2482] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:22.176830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.208120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b707fe-57a8-49fa-bdb8-4e16ee8bc5e0 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.216184 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.216857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.217148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.217433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.218183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:36:22.218498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.220659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.241788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1257/2483] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:36:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:36:22.245895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.261479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97c05-a499-4384-b01c-1c998f0fb118 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.268392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.269177 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.269541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.270397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.271206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.276954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.333473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1227/2484] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:22 2022] POST /identity/v3/projects => generated 379 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:36:22.337285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.348467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04431e58-9235-4e63-87dc-61f0a6fc3a5b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.451010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.452605 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.453219 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.453657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.454399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.456987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.481713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1258/2485] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:22 2022] POST /identity/v3/users => generated 290 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:36:22.486153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.498086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0f3f6-c9d8-45fa-91fe-6308e3c9cc46 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.504008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.505421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.505421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.505567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.506305 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.508274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.518866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1228/2486] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:22.523069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.533706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc75e5-eb11-4dab-b0a6-dbebff3dbee5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.540025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67385005-cc83-4e5d-ac7c-9f20c56ab242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.540780 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67385005-cc83-4e5d-ac7c-9f20c56ab242 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.541072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67385005-cc83-4e5d-ac7c-9f20c56ab242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.541502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67385005-cc83-4e5d-ac7c-9f20c56ab242 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.542234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67385005-cc83-4e5d-ac7c-9f20c56ab242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.544005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67385005-cc83-4e5d-ac7c-9f20c56ab242 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.559553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1259/2487] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:22.560539 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.571139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c22ca2-fb31-4ee7-ad86-8cf178e938ef admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.576826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.578223 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.578509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.579181 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] PATH_INFO: `/v3/projects/98f83ca34fbd4b2c8ec71961d4b947c9/users/1908d906b5a34f8e9b8b0ce82840077e/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.605358 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f83ca34fbd4b2c8ec71961d4b947c9, user_id=1908d906b5a34f8e9b8b0ce82840077e, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.608429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.626018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1229/2488] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:22 2022] PUT /identity/v3/projects/98f83ca34fbd4b2c8ec71961d4b947c9/users/1908d906b5a34f8e9b8b0ce82840077e/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:36:22.639817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.640111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.640392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.683814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] MFA Rules not processed for user `1908d906b5a34f8e9b8b0ce82840077e`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:22.685035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:22.747818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:22.760919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1260/2489] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:22.765625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.853577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d7567-65cb-48c3-9844-66652574af11 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:22.861529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:22.862219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:22.862503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:22.862776 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:22.864672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:22.866735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:22.909805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1230/2490] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:22 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:36:22.914150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:22.936604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3dc28-0e8c-4d5e-adce-9efd0fdd7a32 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.009682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.010678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.010977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.011335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.012152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.014556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:23.047650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1261/2491] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:22 2022] POST /identity/v3/users => generated 297 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:23.048820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:23.064367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf4158-48ed-41ba-890f-711e7fa265a5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.079104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.080161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.081053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.082043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.083005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.085425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:23.102935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1231/2492] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:36:23.107401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:23.118810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b2afca-ec4e-4f72-9451-1fe20a9c75d8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.124289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.125361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.125935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.126526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] PATH_INFO: `/v3/projects/793ceb0bc37f4ab6b6d68b3e812e7b53/users/a61bfb927e6a4728afabe8103d330686/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.145234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=793ceb0bc37f4ab6b6d68b3e812e7b53, user_id=a61bfb927e6a4728afabe8103d330686, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.148938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:23.165628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1262/2493] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:23 2022] PUT /identity/v3/projects/793ceb0bc37f4ab6b6d68b3e812e7b53/users/a61bfb927e6a4728afabe8103d330686/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:23.170415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:23.182929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba497c3-ae31-4614-81ba-486d8be55689 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.225235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:23.259941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749b17d-c945-4dbd-bf4f-98fa600b2684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.260709 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749b17d-c945-4dbd-bf4f-98fa600b2684 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.261507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749b17d-c945-4dbd-bf4f-98fa600b2684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.262057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749b17d-c945-4dbd-bf4f-98fa600b2684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.262893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749b17d-c945-4dbd-bf4f-98fa600b2684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.264839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749b17d-c945-4dbd-bf4f-98fa600b2684 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:23.277811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1232/2494] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:23 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:36:23.282591 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:23.293688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62474b0-aae2-479b-9aad-d47034fb60f3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.299718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.300966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.301051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.301299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] PATH_INFO: `/v3/projects/793ceb0bc37f4ab6b6d68b3e812e7b53/users/a61bfb927e6a4728afabe8103d330686/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.306249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=793ceb0bc37f4ab6b6d68b3e812e7b53, user_id=a61bfb927e6a4728afabe8103d330686, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.309153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:23.323063 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1263/2495] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:23 2022] PUT /identity/v3/projects/793ceb0bc37f4ab6b6d68b3e812e7b53/users/a61bfb927e6a4728afabe8103d330686/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:23.335826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.336166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.336409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.374378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] MFA Rules not processed for user `a61bfb927e6a4728afabe8103d330686`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:23.375887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:23.480007 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1233/2496] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:23.493228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:23.497774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:23.579220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8f2a0-5747-4c7e-bdbf-5a6aa205ab7e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.585243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.586116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.586603 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.587103 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.598672 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.600670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:23.627278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1264/2497] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:23.775156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:23.787380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195c86c-9b96-445d-b0ee-3f24fb5bcfac None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:23.799664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:23.800281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:23.801152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:23.801152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:23.987812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:23.991614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:24.010357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1234/2498] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:24.325572 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:24.328306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:24.408811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:24.414559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036a57cb-830b-434c-86fb-fb27c6a6b469 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:24.419352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:24.420514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:24.420814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:24.421268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:24.423678 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:24.435302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:24.436757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:24.445740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1265/2499] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:24.797524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:24.798959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:24.819835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ab27f-c2b8-4293-ae38-cc8e55177131 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:24.824974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:24.830732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:24.830732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:24.830732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:24.831588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:24.908806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:24.947298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:24.948772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:24.972165 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1235/2500] 10.222.0.45 () {60 vars in 1396 bytes} [Mon Jun 20 06:36:24 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3240 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:25.268145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:25.270295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:25.416062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb361ad-8db3-418b-9a7f-7ed0cd83efcc service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:25.425455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:25.429885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:25.430751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:25.431136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:25.431604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:25.437671 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:25.439171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5539b-1a2d-47fd-965c-3b856f488681 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:25.446365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:25.447201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:25.447763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:25.448170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:25.450627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:25.451961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:25.454440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:25.471290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1266/2501] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:25.569176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:25.571510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:25.617845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1236/2502] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:36:26.668290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:26.689072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1592d-a77b-4e21-b09a-373ffcce5388 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:26.709637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:26.712701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:26.713019 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:26.713379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:26.721962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:26.734610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:26.736869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:26.763725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:26.765283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:26.770814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7210900-2dbc-4a95-8186-6075caac1233 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:26.783779 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:26.786084 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:26.786577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:26.787133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] PATH_INFO: `/v3/users/6490c3d52b5a45aea9236e171ae94770` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:26.791188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6490c3d52b5a45aea9236e171ae94770)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:26.796094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1267/2503] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:26.798241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:26.884249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6490c3d52b5a45aea9236e171ae94770'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:26.895122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6490c3d52b5a45aea9236e171ae94770'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:26.897556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6490c3d52b5a45aea9236e171ae94770'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:26.898368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6490c3d52b5a45aea9236e171ae94770'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:26.922243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6490c3d52b5a45aea9236e171ae94770'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:26.942276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1237/2504] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:26 2022] DELETE /identity/v3/users/6490c3d52b5a45aea9236e171ae94770 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:36:27.046999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:27.048801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:27.070516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e6899f-5d67-4376-b488-c5c958ffd1e3 service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:27.150420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:27.151474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.152018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.152251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] PATH_INFO: `/v3/users/adab7140fc804689af945257511cfded` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.153607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=adab7140fc804689af945257511cfded)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:27.155728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:27.185787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:27.188753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:27.211164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5ea2a4-dd38-4913-b771-fa7b645db439 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:27.217370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:27.218932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.219451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.219738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adab7140fc804689af945257511cfded'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.220022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] PATH_INFO: `/v3/projects/75f7c2d4c0e742cd93e51fee7df062f0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.221665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f7c2d4c0e742cd93e51fee7df062f0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:27.224292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:27.228532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adab7140fc804689af945257511cfded'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.229564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adab7140fc804689af945257511cfded'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.230269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adab7140fc804689af945257511cfded'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.245588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:27.261126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:27.265115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-932be7ac-1f46-4c84-b245-b8286b59ab8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adab7140fc804689af945257511cfded'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.297992 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f7c2d4c0e742cd93e51fee7df062f0', 'request_id': 'req-51e0c243-5a11-40bf-9090-d9c1432655b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.298946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1268/2505] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:27 2022] DELETE /identity/v3/users/adab7140fc804689af945257511cfded => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:27.324914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f7c2d4c0e742cd93e51fee7df062f0', 'request_id': 'req-51e0c243-5a11-40bf-9090-d9c1432655b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.339587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f7c2d4c0e742cd93e51fee7df062f0', 'request_id': 'req-51e0c243-5a11-40bf-9090-d9c1432655b7'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.368459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] The token cache is being invalidate because project 75f7c2d4c0e742cd93e51fee7df062f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:27.369735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.373372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-51e0c243-5a11-40bf-9090-d9c1432655b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.377812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1238/2506] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:27 2022] DELETE /identity/v3/projects/75f7c2d4c0e742cd93e51fee7df062f0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:36:27.384817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.385117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.385397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.420525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:27.426001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:27.567918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1269/2507] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:27.571890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.572349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.572811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.613853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:27.619908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:27.643402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1239/2508] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:36:27.649095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:27.664209 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5a19e9-b26d-4a9f-978e-51d1fef141b9 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:27.677835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:27.679893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.682317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.682819 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.685604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:36:27.686086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:27.690197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:27.720165 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1270/2509] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:36:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:36:27.725916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:27.746489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:27.751938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c27ec5b-f117-4fea-9dfb-476b0d34e98f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:27.752564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:27.772637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87596be-79af-42ca-94cd-188f363caa2b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:27.775323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:27.778260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.779265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.780166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.781014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:27.785423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:27.789009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:27.789639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.790719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.791143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] PATH_INFO: `/v3/projects/cf7ee2aaec40481da0190b662acfde8d` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.794944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf7ee2aaec40481da0190b662acfde8d)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:27.799486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:27.834060 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1240/2510] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:27 2022] POST /identity/v3/projects => generated 397 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:27.838365 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:27.873363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf7ee2aaec40481da0190b662acfde8d', 'request_id': 'req-d42dbf87-c69e-4abb-8818-f19297b513f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.880700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf7ee2aaec40481da0190b662acfde8d', 'request_id': 'req-d42dbf87-c69e-4abb-8818-f19297b513f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.888536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf7ee2aaec40481da0190b662acfde8d', 'request_id': 'req-d42dbf87-c69e-4abb-8818-f19297b513f9'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.918777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] The token cache is being invalidate because project cf7ee2aaec40481da0190b662acfde8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:27.919545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.920601 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d42dbf87-c69e-4abb-8818-f19297b513f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:27.929059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1271/2511] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:27 2022] DELETE /identity/v3/projects/cf7ee2aaec40481da0190b662acfde8d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:27.929202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746bded-9898-4b42-b526-0f5b3dbbe5c3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:27.931376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5a797-39a8-42fc-b65a-b0a15ac9c8e9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.931936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5a797-39a8-42fc-b65a-b0a15ac9c8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.932518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5a797-39a8-42fc-b65a-b0a15ac9c8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.955475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5dc5a797-39a8-42fc-b65a-b0a15ac9c8e9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:27.957005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5dc5a797-39a8-42fc-b65a-b0a15ac9c8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:27.996727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:27.997407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:27.997699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:27.997971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:27.998934 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.000935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.034529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:28.045466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1241/2512] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:27 2022] POST /identity/v3/users => generated 299 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:28.048513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.049352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1272/2513] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:28.052404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:28.058158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.058650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.059111 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.080250 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51887d-ca94-4d6c-9ce8-cfa0c4b33a88 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.085656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:28.087439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:28.092711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.093574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.095339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.095768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.097106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.106915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1273/2514] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:28.106915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.107726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.120095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60626e38-1f08-4226-9c7e-96ed3ae59abd None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.121815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1242/2515] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:28.126025 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.127097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.128230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.134233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.134233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.134233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:36:28.134233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.137266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.139933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bcdc43-950e-4587-b69b-6704a69576f3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.145191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.147496 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.147904 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.148285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.149122 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.150811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.153245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:28.157400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1274/2516] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:36:28.161337 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.170543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1243/2517] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:28.171583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.179239 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682867a3-a751-4284-b29d-d0e0a2f61178 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.182297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565f261-9459-4f92-b460-752da1917e8b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.185693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.187203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.187645 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.187827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216824b-1b74-47cc-a929-ea81994ff28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.188224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.189016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216824b-1b74-47cc-a929-ea81994ff28a admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.189533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216824b-1b74-47cc-a929-ea81994ff28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.189991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216824b-1b74-47cc-a929-ea81994ff28a admin admin] PATH_INFO: `/v3/projects/efb3fbe163cc428b969dfd6a0749c3ba/users/3b4253cdfc5e47b6b85a0d00b223d2d9/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.190310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.193740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.206639 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216824b-1b74-47cc-a929-ea81994ff28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb3fbe163cc428b969dfd6a0749c3ba, user_id=3b4253cdfc5e47b6b85a0d00b223d2d9, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.210088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216824b-1b74-47cc-a929-ea81994ff28a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.224722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1244/2518] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:28 2022] PUT /identity/v3/projects/efb3fbe163cc428b969dfd6a0749c3ba/users/3b4253cdfc5e47b6b85a0d00b223d2d9/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:28.236940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.237333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.237758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.242391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1275/2519] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:28 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:28.254488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.272414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] MFA Rules not processed for user `3b4253cdfc5e47b6b85a0d00b223d2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:28.273781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:28.351416 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1245/2520] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:28.368441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.369870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:28.379519 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0500ff11-b4e8-4c49-bbe9-1a78f50e3af6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.386991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.387816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.388177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.388657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.389455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.391691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.426785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1276/2521] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:28 2022] POST /identity/v3/users => generated 292 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:28.427909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.436286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65824434-8747-4e00-9e67-2a08aa770c55 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.440704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c4fd1c-9e2d-4dec-bade-e50fdfd4bf76 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.440881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.441543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.441832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.442215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.447115 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.447976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.448331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.448681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.449799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.451842 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.452843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.454344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.465404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1246/2522] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:28.472025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1277/2523] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:28.473138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.485938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07361a-9d6d-4818-b08f-7244a0388d16 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.494273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea041714-74ba-4993-9767-ba6cacc88cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.495009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea041714-74ba-4993-9767-ba6cacc88cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.495396 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea041714-74ba-4993-9767-ba6cacc88cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.495748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea041714-74ba-4993-9767-ba6cacc88cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.496627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea041714-74ba-4993-9767-ba6cacc88cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.498436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea041714-74ba-4993-9767-ba6cacc88cd0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.513994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1247/2524] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:28.516064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.529927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb2966f-a50a-47d9-85cd-e85dc9eceac1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.535680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.536724 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.537201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.538392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] PATH_INFO: `/v3/projects/a1d7a585b8244bc5b2b2d8642b9160d0/users/09befefeb2e84ae79e7437b30cb3d908/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.561196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1d7a585b8244bc5b2b2d8642b9160d0, user_id=09befefeb2e84ae79e7437b30cb3d908, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.564302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.582283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1278/2525] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:28 2022] PUT /identity/v3/projects/a1d7a585b8244bc5b2b2d8642b9160d0/users/09befefeb2e84ae79e7437b30cb3d908/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:28.587702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a627c1-c121-436b-a38b-749b164e194c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.588002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a627c1-c121-436b-a38b-749b164e194c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.588283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a627c1-c121-436b-a38b-749b164e194c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.612204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-11a627c1-c121-436b-a38b-749b164e194c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:28.613487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-11a627c1-c121-436b-a38b-749b164e194c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:28.669894 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:28.686469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1248/2526] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:36:28.693233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:28.695716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:28.755108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f72768-a30c-4347-b3aa-7682cdc3355a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:28.759628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1177e0f-0e50-4e85-9daa-1f4aee7a5a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:28.760375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1177e0f-0e50-4e85-9daa-1f4aee7a5a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:28.760735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1177e0f-0e50-4e85-9daa-1f4aee7a5a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:28.761088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1177e0f-0e50-4e85-9daa-1f4aee7a5a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:28.784733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1177e0f-0e50-4e85-9daa-1f4aee7a5a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:28.786498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1177e0f-0e50-4e85-9daa-1f4aee7a5a59 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:28.803745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1279/2527] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:30.727142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a627c1-c121-436b-a38b-749b164e194c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:30.743000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a627c1-c121-436b-a38b-749b164e194c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:30.748850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:30.749577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:30.749933 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:30.750219 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:30.752569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:30.777078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:30.778551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:30.790437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1249/2528] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:34.637376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b0931-e765-4f6a-a25a-62988d9239ba None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:34.637928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b0931-e765-4f6a-a25a-62988d9239ba None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:34.637928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b0931-e765-4f6a-a25a-62988d9239ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:34.667008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:34.676299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3b3b0931-e765-4f6a-a25a-62988d9239ba None None] MFA Rules not processed for user `09befefeb2e84ae79e7437b30cb3d908`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:34.676299 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3b3b0931-e765-4f6a-a25a-62988d9239ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:34.796451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1280/2529] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:34.932332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:34.933376 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:35.076817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11941df7-33cc-4840-9768-411ddc98b1d0 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:35.082402 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:35.083086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:35.083445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:35.083727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:35.091946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:35.093534 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:35.102890 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1250/2530] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:37.061578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259229cd-8a86-4d13-aa95-7ed45f36d211 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:37.062118 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259229cd-8a86-4d13-aa95-7ed45f36d211 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:37.062399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259229cd-8a86-4d13-aa95-7ed45f36d211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:37.081665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-259229cd-8a86-4d13-aa95-7ed45f36d211 None None] MFA Rules not processed for user `09befefeb2e84ae79e7437b30cb3d908`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:37.082913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-259229cd-8a86-4d13-aa95-7ed45f36d211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:37.091320 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1281/2531] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:38.273070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:38.274609 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:38.295718 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cba5ff-ee1d-48da-87a6-29038dcc802f service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:38.305246 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:38.306441 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:38.306939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:38.307445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] PATH_INFO: `/v3/users/1908d906b5a34f8e9b8b0ce82840077e` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:38.310773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=1908d906b5a34f8e9b8b0ce82840077e)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:38.314449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:38.422846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1908d906b5a34f8e9b8b0ce82840077e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.444576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1908d906b5a34f8e9b8b0ce82840077e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.445997 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1908d906b5a34f8e9b8b0ce82840077e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.446622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1908d906b5a34f8e9b8b0ce82840077e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.480681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1908d906b5a34f8e9b8b0ce82840077e'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.501168 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1251/2532] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:38 2022] DELETE /identity/v3/users/1908d906b5a34f8e9b8b0ce82840077e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:38.505448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:38.506848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:38.510615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:38.569940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:38.571435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:38.681183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:38.682635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:38.765033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:38.777170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1282/2533] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:38.797365 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:38.800942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:38.824097 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7136b72-1d43-4ce4-b732-5693555ee6ab admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:38.834288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:38.835194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:38.835511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:38.835798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] PATH_INFO: `/v3/users/3b4253cdfc5e47b6b85a0d00b223d2d9` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:38.840505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b4253cdfc5e47b6b85a0d00b223d2d9)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:38.844586 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:38.940016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a2865-a32c-4bd0-93c8-635e94f17838 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:38.945406 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bda08c-2b09-48c6-ad94-c6f223e34e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:38.946446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bda08c-2b09-48c6-ad94-c6f223e34e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:38.947367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bda08c-2b09-48c6-ad94-c6f223e34e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:38.947761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bda08c-2b09-48c6-ad94-c6f223e34e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:38.961240 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bda08c-2b09-48c6-ad94-c6f223e34e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:38.963933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bda08c-2b09-48c6-ad94-c6f223e34e4f service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:38.966499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b4253cdfc5e47b6b85a0d00b223d2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.982238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b4253cdfc5e47b6b85a0d00b223d2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.982985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b4253cdfc5e47b6b85a0d00b223d2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.983699 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b4253cdfc5e47b6b85a0d00b223d2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:38.992511 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1283/2534] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:39.018808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b4253cdfc5e47b6b85a0d00b223d2d9'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.061770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1252/2535] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:38 2022] DELETE /identity/v3/users/3b4253cdfc5e47b6b85a0d00b223d2d9 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:39.063689 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:39.064318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:39.064850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:39.096890 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:39.098194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:39.157262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:39.211555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1284/2536] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:39.221814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:39.223390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:39.333691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c34ee8-802e-4b54-8e4a-4890b0cb7566 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:39.341149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:39.343083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:39.343083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:39.343286 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:39.356797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:39.358952 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:39.383507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1253/2537] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:39.404623 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:39.408763 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:39.439825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24197bc0-00e7-4708-9501-8b5964bc00f8 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:39.451737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:39.452651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:39.452944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:39.453224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] PATH_INFO: `/v3/projects/98f83ca34fbd4b2c8ec71961d4b947c9` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:39.454687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f83ca34fbd4b2c8ec71961d4b947c9)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:39.456698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:39.539706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f83ca34fbd4b2c8ec71961d4b947c9', 'request_id': 'req-13abd87a-fa36-42ee-91ba-5878cc18799a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.546661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f83ca34fbd4b2c8ec71961d4b947c9', 'request_id': 'req-13abd87a-fa36-42ee-91ba-5878cc18799a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.586313 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f83ca34fbd4b2c8ec71961d4b947c9', 'request_id': 'req-13abd87a-fa36-42ee-91ba-5878cc18799a'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.603001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] The token cache is being invalidate because project 98f83ca34fbd4b2c8ec71961d4b947c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:39.603507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.605330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.617013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1285/2538] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:39 2022] DELETE /identity/v3/projects/98f83ca34fbd4b2c8ec71961d4b947c9 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:39.622183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:39.658620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34bce8c-258c-4731-9b5e-e22d2b06952b service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:39.794125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:39.795582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:39.796071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:39.796545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] PATH_INFO: `/v3/users/a61bfb927e6a4728afabe8103d330686` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:39.798052 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61bfb927e6a4728afabe8103d330686)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:39.801033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:39.804401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:39.920974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61bfb927e6a4728afabe8103d330686'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.933003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:39.934643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:39.936136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61bfb927e6a4728afabe8103d330686'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.943791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61bfb927e6a4728afabe8103d330686'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.945157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61bfb927e6a4728afabe8103d330686'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:39.977380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61bfb927e6a4728afabe8103d330686'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.016137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abd87a-fa36-42ee-91ba-5878cc18799a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:40.019151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1254/2539] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:39 2022] DELETE /identity/v3/users/a61bfb927e6a4728afabe8103d330686 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:40.184263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:40.185319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:40.185608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:40.186000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] PATH_INFO: `/v3/projects/efb3fbe163cc428b969dfd6a0749c3ba` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:40.195638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb3fbe163cc428b969dfd6a0749c3ba)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:40.200032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:40.290851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb3fbe163cc428b969dfd6a0749c3ba', 'request_id': 'req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.300837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb3fbe163cc428b969dfd6a0749c3ba', 'request_id': 'req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.321992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb3fbe163cc428b969dfd6a0749c3ba', 'request_id': 'req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.325210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:40.347006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] The token cache is being invalidate because project efb3fbe163cc428b969dfd6a0749c3ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:40.347489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.348445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-4f4a0cf9-7fcd-42d5-a677-4e175fa58015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.351046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1286/2540] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:39 2022] DELETE /identity/v3/projects/efb3fbe163cc428b969dfd6a0749c3ba => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:36:40.662922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:40.664598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:40.679739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e285e5-ef74-4e4f-a558-4b13e18d8c11 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:40.760649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:40.762629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:40.762963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:40.763269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] PATH_INFO: `/v3/projects/793ceb0bc37f4ab6b6d68b3e812e7b53` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:40.764543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=793ceb0bc37f4ab6b6d68b3e812e7b53)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:40.766714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:40.823522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '793ceb0bc37f4ab6b6d68b3e812e7b53', 'request_id': 'req-4209d772-2b68-44f1-8633-3820b42c2e7d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.831201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '793ceb0bc37f4ab6b6d68b3e812e7b53', 'request_id': 'req-4209d772-2b68-44f1-8633-3820b42c2e7d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.839711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '793ceb0bc37f4ab6b6d68b3e812e7b53', 'request_id': 'req-4209d772-2b68-44f1-8633-3820b42c2e7d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.859808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] The token cache is being invalidate because project 793ceb0bc37f4ab6b6d68b3e812e7b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:40.861146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.862034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-4209d772-2b68-44f1-8633-3820b42c2e7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:40.867046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1255/2541] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:40 2022] DELETE /identity/v3/projects/793ceb0bc37f4ab6b6d68b3e812e7b53 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:40.868777 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:40.869257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:40.869358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:40.896036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:40.898221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:41.007290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1287/2542] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:41.013507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.013796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.014082 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.048938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:41.050188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:41.071298 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1256/2543] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:41.075488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.087698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e126f7-4263-4e99-ba59-7dc82c3694a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.107698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.110765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.110765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.110765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.113326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:36:41.113652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.125141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.142251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1288/2544] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:36:41.143432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.151459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:41.158503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c981be3-af64-43b7-b424-ed5882b698f6 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.178140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.179482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.179853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.180236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.181016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.183074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.213751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1257/2545] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:41 2022] POST /identity/v3/projects => generated 369 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:41.217489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.235027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eca70f-8eba-4880-b50f-1eea7d2a351d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.321079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.322091 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.322549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.323650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.324490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.326685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.356925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1289/2546] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:41 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:41.360658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.374150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd215673-d317-4944-a23a-248a13c8d852 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.380317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.381353 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.381826 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.382306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.383317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.385306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.399391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1258/2547] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:41.403281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.414789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae120b6a-980a-40b5-a42b-05341d00c241 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.419815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ef86d-041c-4914-8ea4-9348726c6b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.421031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ef86d-041c-4914-8ea4-9348726c6b8e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.421410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ef86d-041c-4914-8ea4-9348726c6b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.421758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ef86d-041c-4914-8ea4-9348726c6b8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.422559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ef86d-041c-4914-8ea4-9348726c6b8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.424281 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ef86d-041c-4914-8ea4-9348726c6b8e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.455521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1290/2548] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:41.459570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.472472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fac0db-fb9b-47bd-ae70-5550e250816f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.478483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.479440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.479735 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.480019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] PATH_INFO: `/v3/projects/f62b7ac6b3ee40618c235cdb1a104543/users/8d39559e74d94b88b37d76a6f30891c6/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.502771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f62b7ac6b3ee40618c235cdb1a104543, user_id=8d39559e74d94b88b37d76a6f30891c6, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.505615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.520837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1259/2549] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:41 2022] PUT /identity/v3/projects/f62b7ac6b3ee40618c235cdb1a104543/users/8d39559e74d94b88b37d76a6f30891c6/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:41.532513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.533101 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.533514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.567703 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] MFA Rules not processed for user `8d39559e74d94b88b37d76a6f30891c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:41.569100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:41.629779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:41.645469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1291/2550] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:41 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:36:41.650311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.742161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fab657b-1425-438f-acbf-a43c6519b456 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.748570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.749464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.749916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.750420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.751331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.753471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.787437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1260/2551] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:41 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:41.791345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.833323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e3ced-6604-49b8-94a6-8ffca2475354 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.916307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.916989 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.917272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.917550 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.918257 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.920488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.942924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1292/2552] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:41 2022] POST /identity/v3/users => generated 290 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:36:41.951580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:41.964446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0cb7bd-1a31-417b-afdb-265fb14d51d8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:41.969809 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:41.970487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:41.970773 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:41.971049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:41.971813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:41.974293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:41.986510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1261/2553] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:41.990565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:42.007557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df586a9f-aaf7-43c1-83bf-a29cd1860551 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:42.012434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:42.013333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:42.013665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:42.013888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] PATH_INFO: `/v3/projects/4f816c2e3fec4c828d9cb898393c7636/users/813d517719124168933f25383e861ce8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:42.029610 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f816c2e3fec4c828d9cb898393c7636, user_id=813d517719124168933f25383e861ce8, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:42.032334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:42.047574 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1293/2554] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:41 2022] PUT /identity/v3/projects/4f816c2e3fec4c828d9cb898393c7636/users/813d517719124168933f25383e861ce8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:42.051459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:42.061438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57127108-64dd-4835-973f-f76fc86eb59b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:42.117604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc5447-af51-4d64-ab28-8416f22f6b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:42.118194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5447-af51-4d64-ab28-8416f22f6b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:42.118483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5447-af51-4d64-ab28-8416f22f6b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:42.118764 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc5447-af51-4d64-ab28-8416f22f6b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:42.119538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc5447-af51-4d64-ab28-8416f22f6b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:42.121128 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc5447-af51-4d64-ab28-8416f22f6b69 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:42.131882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1262/2555] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:42.136029 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:42.146242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6f5dde-af32-41c8-bc7e-7f48a3383ce7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:42.151396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:42.152272 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:42.152553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:42.152831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] PATH_INFO: `/v3/projects/4f816c2e3fec4c828d9cb898393c7636/users/813d517719124168933f25383e861ce8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:42.154797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f816c2e3fec4c828d9cb898393c7636, user_id=813d517719124168933f25383e861ce8, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:42.157487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:42.168880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1294/2556] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:42 2022] PUT /identity/v3/projects/4f816c2e3fec4c828d9cb898393c7636/users/813d517719124168933f25383e861ce8/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:42.229194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:42.229585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:42.230010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:42.261715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] MFA Rules not processed for user `813d517719124168933f25383e861ce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:42.263061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:42.272126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:42.365432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1263/2557] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:42 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:42.378133 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:42.381503 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:42.462203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f9ce4-16a5-4e81-9c55-2dcd6e95da2f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:42.468062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b2b9e1-01c6-4cf9-86da-c208141df2c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:42.468838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2b9e1-01c6-4cf9-86da-c208141df2c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:42.469230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2b9e1-01c6-4cf9-86da-c208141df2c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:42.469673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b2b9e1-01c6-4cf9-86da-c208141df2c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:42.480444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2b9e1-01c6-4cf9-86da-c208141df2c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:42.482109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b2b9e1-01c6-4cf9-86da-c208141df2c9 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:42.495506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1295/2558] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:42 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:36:44.048378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:44.121315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c841e-f983-456e-84fb-5b30687f2b26 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:44.126347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:44.127238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:44.127555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:44.127840 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] PATH_INFO: `/v3/users/8d39559e74d94b88b37d76a6f30891c6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:44.129045 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d39559e74d94b88b37d76a6f30891c6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:44.131577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:44.189693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d39559e74d94b88b37d76a6f30891c6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.197719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d39559e74d94b88b37d76a6f30891c6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.198573 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d39559e74d94b88b37d76a6f30891c6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.199195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d39559e74d94b88b37d76a6f30891c6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.214144 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d39559e74d94b88b37d76a6f30891c6'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.242694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1264/2559] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:44 2022] DELETE /identity/v3/users/8d39559e74d94b88b37d76a6f30891c6 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:44.247897 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:44.248381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:44.248487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:44.269095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:44.270509 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:44.325483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:44.348162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1296/2560] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:44.358892 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:44.360401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:44.387269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:44.419300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de54f6-c1a0-429d-8123-300dd171d101 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:44.422919 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:44.423595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:44.423883 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:44.424163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:44.433608 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:44.435026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:44.444463 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1265/2561] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:44.722347 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:44.725648 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:44.743563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd150af4-c38f-4009-9a6b-9079cbb8326d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:44.749130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:44.750012 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:44.750375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:44.750785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] PATH_INFO: `/v3/projects/f62b7ac6b3ee40618c235cdb1a104543` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:44.752023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f62b7ac6b3ee40618c235cdb1a104543)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:44.755083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:44.846904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f62b7ac6b3ee40618c235cdb1a104543', 'request_id': 'req-a009f4d1-e95d-4812-8abc-d3e36691c842'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.854902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f62b7ac6b3ee40618c235cdb1a104543', 'request_id': 'req-a009f4d1-e95d-4812-8abc-d3e36691c842'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.864031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f62b7ac6b3ee40618c235cdb1a104543', 'request_id': 'req-a009f4d1-e95d-4812-8abc-d3e36691c842'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.888268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] The token cache is being invalidate because project f62b7ac6b3ee40618c235cdb1a104543 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:44.888578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.889231 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:44.891038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1297/2562] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:44 2022] DELETE /identity/v3/projects/f62b7ac6b3ee40618c235cdb1a104543 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:44.894965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:44.911403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3baffd2-6a49-4ca7-b061-43d6bc52b324 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:44.994648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:44.996583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:44.997233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:44.998004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] PATH_INFO: `/v3/users/813d517719124168933f25383e861ce8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:44.999753 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813d517719124168933f25383e861ce8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:45.002535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:45.075610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813d517719124168933f25383e861ce8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.091567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813d517719124168933f25383e861ce8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.093036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813d517719124168933f25383e861ce8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.093943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813d517719124168933f25383e861ce8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.112178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-885f2c3e-4523-4a1e-865c-a90546f38a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813d517719124168933f25383e861ce8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.130274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1266/2563] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:36:44 2022] DELETE /identity/v3/users/813d517719124168933f25383e861ce8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:45.418609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:45.420130 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:45.446345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009f4d1-e95d-4812-8abc-d3e36691c842 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:45.503974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:45.504838 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.505119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.505491 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] PATH_INFO: `/v3/projects/4f816c2e3fec4c828d9cb898393c7636` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.507203 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f816c2e3fec4c828d9cb898393c7636)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:45.509139 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:45.556625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f816c2e3fec4c828d9cb898393c7636', 'request_id': 'req-ed0694b5-48b8-4a08-94ab-f7947898e79b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.562140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f816c2e3fec4c828d9cb898393c7636', 'request_id': 'req-ed0694b5-48b8-4a08-94ab-f7947898e79b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.569362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f816c2e3fec4c828d9cb898393c7636', 'request_id': 'req-ed0694b5-48b8-4a08-94ab-f7947898e79b'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.577377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:45.583468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] The token cache is being invalidate because project 4f816c2e3fec4c828d9cb898393c7636 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:36:45.584258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.585318 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ed0694b5-48b8-4a08-94ab-f7947898e79b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:36:45.587041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1298/2564] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:36:45 2022] DELETE /identity/v3/projects/4f816c2e3fec4c828d9cb898393c7636 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:45.591994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.592290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.592576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.610958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:45.612199 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:45.654701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:45.676071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1267/2565] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:45.684536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.684828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.685156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.705737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:45.707226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:45.715705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1299/2566] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:36:45.719568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:45.730315 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0cf0a-69e7-4d0a-b610-3ca08fd9614f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:45.735166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:45.735874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.736171 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.736453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.737192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:36:45.737503 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:45.739119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:45.752667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1268/2567] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:36:45.756865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:45.766504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c849a7-511e-449d-92b5-511ede98f34e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:45.771479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:45.772136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.772422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.772701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.773451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:45.775394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:45.802884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1300/2568] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:45 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:36:45.807053 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:45.817860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15705b95-45b1-4d6d-b26e-9de8a16646d3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:45.912672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:45.913346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.913630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.913920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.914649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:45.916915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:45.947299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1269/2569] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:45 2022] POST /identity/v3/users => generated 285 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:45.950001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:45.960964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52772e3d-495f-4845-aafb-5fb9df782369 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:45.966362 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:45.967403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:45.967529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:45.967938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:45.968948 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:45.970685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:45.992711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1301/2570] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:45.997108 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:46.012309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4122e135-0ed8-4e1a-b9c4-50582837d7f9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:46.018243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:46.018936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:46.019577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:46.019577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:46.020900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:46.023191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:46.035598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1270/2571] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:46.040273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:46.054734 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b0eeb-ef2d-4a45-bafc-6fcce2a72f58 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:46.074889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14601ab4-eece-4dfc-ab27-d1604599712c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:46.075837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601ab4-eece-4dfc-ab27-d1604599712c admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:46.076126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601ab4-eece-4dfc-ab27-d1604599712c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:46.076407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14601ab4-eece-4dfc-ab27-d1604599712c admin admin] PATH_INFO: `/v3/projects/37757c59aaeb41fdbf2790cc1be0cde2/users/d25876ee10594220af1926ebe98ee927/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:46.086505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:46.088733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:46.111316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14601ab4-eece-4dfc-ab27-d1604599712c admin admin] RBAC: Authorizing `identity:create_grant(project_id=37757c59aaeb41fdbf2790cc1be0cde2, user_id=d25876ee10594220af1926ebe98ee927, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:46.114336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14601ab4-eece-4dfc-ab27-d1604599712c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:46.135857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1302/2572] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:46 2022] PUT /identity/v3/projects/37757c59aaeb41fdbf2790cc1be0cde2/users/d25876ee10594220af1926ebe98ee927/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:46.141305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:46.142199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:46.142635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:46.170839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:46.173016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:46.194153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28479cd-90e0-4909-9ae5-89f6beb18b50 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:46.253144 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:46.264635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1303/2573] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:46.275514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:46.279416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:46.280019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:46.280830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:46.281198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:46.281569 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:46.283742 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:46.299897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:46.301632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5438ab4f-8066-4fc9-9ca6-2955b2a9b24a service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:46.315188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1271/2574] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:36:46.389999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a4079-6658-40d4-b48f-d40b76d8024e None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:46.395547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:46.397259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:46.397716 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:46.398138 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:46.439795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:46.445312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:46.456918 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1304/2575] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:47.295100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.295636 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.295725 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.336655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] MFA Rules not processed for user `d25876ee10594220af1926ebe98ee927`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:47.337867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:47.420058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1272/2576] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:47 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:47.424526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:47.435067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c990c-03c6-424c-9bd9-177480d5c635 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:47.439845 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:47.440512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.440799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.441075 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.441776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:47.443778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:47.477376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1305/2577] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:36:47 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:36:47.479482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:47.494135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f76ee-20aa-4bd7-8dc8-36a4dfac286c None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:47.570189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:47.570970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.571327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.571720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.572425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:47.574403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:47.598120 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1273/2578] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:36:47 2022] POST /identity/v3/users => generated 292 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:36:47.602615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:47.613646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d16523a-d18b-4f80-b1c9-0ae22fe8bc40 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:47.619836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:47.620828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.621187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.621683 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.622488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:47.625885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:47.643302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1306/2579] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:36:47.647184 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:47.659949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9283405-972b-409d-95a1-e36eaea65f85 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:47.665833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:47.667182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.667727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.668137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] PATH_INFO: `/v3/projects/b52bd91897394dfe9d463fea4c888ddd/users/9145b4d4132d457ca8a10e986f9762ff/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.687217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52bd91897394dfe9d463fea4c888ddd, user_id=9145b4d4132d457ca8a10e986f9762ff, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:47.690027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:47.704468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1274/2580] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:47 2022] PUT /identity/v3/projects/b52bd91897394dfe9d463fea4c888ddd/users/9145b4d4132d457ca8a10e986f9762ff/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:47.708927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:47.716382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:47.726461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c4b602-2567-4d80-8ca7-3ef7556b9ddb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:47.796726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a1352-4913-4487-9546-f67f28b775c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:47.797396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a1352-4913-4487-9546-f67f28b775c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.797680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a1352-4913-4487-9546-f67f28b775c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.797956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a1352-4913-4487-9546-f67f28b775c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.798707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a1352-4913-4487-9546-f67f28b775c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:47.800343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a1352-4913-4487-9546-f67f28b775c3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:47.810557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1307/2581] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:36:47 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:36:47.814509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:47.829959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a695a-2ffe-4749-8c00-a3461d130e7b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:47.835359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:47.836243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:47.836528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:47.836805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] PATH_INFO: `/v3/projects/b52bd91897394dfe9d463fea4c888ddd/users/9145b4d4132d457ca8a10e986f9762ff/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:47.838777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b52bd91897394dfe9d463fea4c888ddd, user_id=9145b4d4132d457ca8a10e986f9762ff, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:47.841412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:47.853582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1275/2582] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:36:47 2022] PUT /identity/v3/projects/b52bd91897394dfe9d463fea4c888ddd/users/9145b4d4132d457ca8a10e986f9762ff/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:36:48.597835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:48.598849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:48.599174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:48.660119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] MFA Rules not processed for user `9145b4d4132d457ca8a10e986f9762ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:36:48.662270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:48.791830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1308/2583] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:36:48 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:36:48.812046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:48.814123 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:48.826149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:48.828352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:48.867845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:48.934991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af7ed43-3e8b-4e82-84c3-6ae8fe86c19e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:48.940348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:48.940996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:48.941176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:48.941439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:48.942229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0e2307-e1f8-4599-a73a-d6cde10b4d81 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:48.943613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:48.948974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:48.950428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:48.951089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:48.951089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:49.003307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:36:49.029640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:49.031563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:49.044259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1309/2584] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:48 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:49.098310 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:49.100834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:49.125541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1276/2585] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:50.450497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:50.459726 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ee6797-6c70-4e6c-984c-0c8a2cf5b84f service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:50.463680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:50.464500 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:50.464975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:50.465270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:50.479480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:50.482482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:50.499166 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1310/2586] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:50 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:36:52.666963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:36:52.766866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8f3d1-1ae4-44b7-9a35-61400ba200a0 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:36:52.773090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:36:52.773755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:36:52.774048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:36:52.774410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:36:52.776249 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:36:52.791196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:36:52.792928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:36:52.805294 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1277/2587] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:36:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:37:00.491899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=15 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:02.800442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:03.729626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:03.798659 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad974e0-78ab-46f0-9517-a7873bf5dfc3 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:03.803241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:03.804143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:03.804424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:03.804817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] PATH_INFO: `/v3/users/d25876ee10594220af1926ebe98ee927` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:03.806011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d25876ee10594220af1926ebe98ee927)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:03.807974 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:03.861475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd25876ee10594220af1926ebe98ee927'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:03.868952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd25876ee10594220af1926ebe98ee927'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:03.869469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd25876ee10594220af1926ebe98ee927'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:03.870084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd25876ee10594220af1926ebe98ee927'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:03.882978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd25876ee10594220af1926ebe98ee927'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:03.896673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1311/2588] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:37:03 2022] DELETE /identity/v3/users/d25876ee10594220af1926ebe98ee927 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:04.076410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:04.095891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde17adc-c968-4b2a-a4b6-6488977383cf service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:04.163788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:04.164717 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.165005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.165338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] PATH_INFO: `/v3/projects/37757c59aaeb41fdbf2790cc1be0cde2` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.166561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] RBAC: Authorizing `identity:delete_project(project_id=37757c59aaeb41fdbf2790cc1be0cde2)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:04.168607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:04.214188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37757c59aaeb41fdbf2790cc1be0cde2', 'request_id': 'req-6eebd485-4447-4d19-bdcf-5455206ae59b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.219836 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37757c59aaeb41fdbf2790cc1be0cde2', 'request_id': 'req-6eebd485-4447-4d19-bdcf-5455206ae59b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.226861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37757c59aaeb41fdbf2790cc1be0cde2', 'request_id': 'req-6eebd485-4447-4d19-bdcf-5455206ae59b'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.240359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] The token cache is being invalidate because project 37757c59aaeb41fdbf2790cc1be0cde2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:37:04.240648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.241264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.243033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1278/2589] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:37:04 2022] DELETE /identity/v3/projects/37757c59aaeb41fdbf2790cc1be0cde2 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:04.246964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:04.248557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:04.261872 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2681cc4f-ee4a-474f-94fb-65e1d4ac0c09 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:04.318558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:04.319485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.319787 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.320053 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] PATH_INFO: `/v3/users/9145b4d4132d457ca8a10e986f9762ff` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.321245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9145b4d4132d457ca8a10e986f9762ff)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:04.323216 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:04.345543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:04.373433 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9145b4d4132d457ca8a10e986f9762ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.380796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9145b4d4132d457ca8a10e986f9762ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.381479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9145b4d4132d457ca8a10e986f9762ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.382098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9145b4d4132d457ca8a10e986f9762ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.394634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2292e506-35a2-4862-930f-aa8a2a346e4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9145b4d4132d457ca8a10e986f9762ff'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.413627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1312/2590] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:37:04 2022] DELETE /identity/v3/users/9145b4d4132d457ca8a10e986f9762ff => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:04.595011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:04.596610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:04.613312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebd485-4447-4d19-bdcf-5455206ae59b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:04.672604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:04.673390 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.673672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.673946 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] PATH_INFO: `/v3/projects/b52bd91897394dfe9d463fea4c888ddd` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.675121 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b52bd91897394dfe9d463fea4c888ddd)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:04.677141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:04.712297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:04.726419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b52bd91897394dfe9d463fea4c888ddd', 'request_id': 'req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.731939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b52bd91897394dfe9d463fea4c888ddd', 'request_id': 'req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.738227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b52bd91897394dfe9d463fea4c888ddd', 'request_id': 'req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.751648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] The token cache is being invalidate because project b52bd91897394dfe9d463fea4c888ddd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:37:04.751936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.752545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-0bc66ad3-2ed9-43d5-a6af-85339edaa6ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:04.754341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1279/2591] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:37:04 2022] DELETE /identity/v3/projects/b52bd91897394dfe9d463fea4c888ddd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:04.759560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.760059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.760140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.778275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:04.779608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:04.848884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1313/2592] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:04.850866 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.851166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.851791 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.871855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:04.873077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:04.881654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1280/2593] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:04.885996 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:04.896420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d7e85-e35b-4fa4-a6c5-62abf10f5c31 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:04.900951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:04.901604 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.901887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.902161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.902906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:37:04.903212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:04.904875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:04.916688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1314/2594] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:37:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:37:04.920913 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:04.930881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352382c-4fa5-4a32-ab79-3bb20bf33881 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:04.935525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:04.936180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:04.936467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:04.936744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:04.937438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:04.939361 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:04.965096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1281/2595] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:37:04 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:37:04.969220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:04.979475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b08f94-ef08-4bdd-8247-86a0b4cd56e8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:05.038027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:05.038027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:05.038027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:05.038027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:05.038428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:05.039005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:05.062670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1315/2596] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:37:04 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:37:05.063884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:05.074678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f659b-ad2e-4928-9bd4-159c61f2de9d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:05.079204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:05.079914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:05.080183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:05.080464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:05.081192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:05.082819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:05.093576 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1282/2597] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:37:05.097470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:05.108060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066521e5-b1b5-4927-ac30-58a2c5a68b2b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:05.112794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cce420b-c370-4751-a980-3523fa012d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:05.113453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce420b-c370-4751-a980-3523fa012d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:05.113736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce420b-c370-4751-a980-3523fa012d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:05.114542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cce420b-c370-4751-a980-3523fa012d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:05.115296 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce420b-c370-4751-a980-3523fa012d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:05.116935 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cce420b-c370-4751-a980-3523fa012d47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:05.130729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1316/2598] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:37:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:37:05.132183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:05.142364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d6ee4-914b-44c1-8807-e5df3d0fba10 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:05.147059 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:05.147982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:05.148271 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:05.148555 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] PATH_INFO: `/v3/projects/6c9e6e5d3f7045bcb2eece2179de9e66/users/ac94953057e14fa792df4554a24a3138/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:05.164561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9e6e5d3f7045bcb2eece2179de9e66, user_id=ac94953057e14fa792df4554a24a3138, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:05.167201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:05.182087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1283/2599] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:37:05 2022] PUT /identity/v3/projects/6c9e6e5d3f7045bcb2eece2179de9e66/users/ac94953057e14fa792df4554a24a3138/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:05.193285 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:05.193607 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:05.193888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:05.200722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:05.221731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] MFA Rules not processed for user `ac94953057e14fa792df4554a24a3138`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:05.222949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:05.289153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1317/2600] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:05.299860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:05.302198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:05.359427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d19928f-698d-4e96-b085-497f96be7170 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:05.363094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87e554-00d0-42cf-a223-4685fead15c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:05.363787 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e554-00d0-42cf-a223-4685fead15c2 service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:05.364622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e554-00d0-42cf-a223-4685fead15c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:05.364907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e554-00d0-42cf-a223-4685fead15c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:05.376034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87e554-00d0-42cf-a223-4685fead15c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:05.378037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87e554-00d0-42cf-a223-4685fead15c2 service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:05.387972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1284/2601] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:37:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:37:15.286457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=30 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:15.383843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=65,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:21.900828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:21.964147 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c1f60-3826-4932-a891-02ead12e9c0c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:21.968828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:21.969736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:21.970020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:21.970297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] PATH_INFO: `/v3/users/ac94953057e14fa792df4554a24a3138` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:21.971620 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac94953057e14fa792df4554a24a3138)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:21.973546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.020834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac94953057e14fa792df4554a24a3138'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.028379 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac94953057e14fa792df4554a24a3138'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.029068 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac94953057e14fa792df4554a24a3138'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.029694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac94953057e14fa792df4554a24a3138'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.042164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac94953057e14fa792df4554a24a3138'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.055506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1318/2602] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:37:21 2022] DELETE /identity/v3/users/ac94953057e14fa792df4554a24a3138 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:22.060117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.060613 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.060713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.078208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:22.079411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:22.139765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1285/2603] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:37:22.149968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.151479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:22.209736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467e1d6-d94e-4c89-92ca-5ff4626d8c78 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:22.213423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbb761e-2a22-46e4-b942-5c8693c77632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:22.214071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb761e-2a22-46e4-b942-5c8693c77632 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.214353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb761e-2a22-46e4-b942-5c8693c77632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.214628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbb761e-2a22-46e4-b942-5c8693c77632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.223750 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb761e-2a22-46e4-b942-5c8693c77632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:22.225976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbb761e-2a22-46e4-b942-5c8693c77632 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.235676 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1319/2604] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:37:22.523975 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.525684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:22.544218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5f4f8-4b3a-4511-b223-6680ec4c5661 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:22.557234 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:22.558109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.558392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.558665 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] PATH_INFO: `/v3/projects/6c9e6e5d3f7045bcb2eece2179de9e66` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.559876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9e6e5d3f7045bcb2eece2179de9e66)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:22.561803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.609187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9e6e5d3f7045bcb2eece2179de9e66', 'request_id': 'req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.615006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9e6e5d3f7045bcb2eece2179de9e66', 'request_id': 'req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.621352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9e6e5d3f7045bcb2eece2179de9e66', 'request_id': 'req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.634744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] The token cache is being invalidate because project 6c9e6e5d3f7045bcb2eece2179de9e66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:37:22.635035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.635698 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f7dac3a7-6db2-4fd0-b944-787c1652b6bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:22.637273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1286/2605] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:37:22 2022] DELETE /identity/v3/projects/6c9e6e5d3f7045bcb2eece2179de9e66 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:37:22.642907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.643198 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.643521 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.661573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:22.662754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:22.680394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:22.727813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1320/2606] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:22.735637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.736015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.736363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.754950 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:22.756814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:22.764820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1287/2607] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:37:22.769090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.779221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0e31e9-29f8-4633-ab85-6754fb646ca6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:22.783832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:22.784577 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.784884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.785163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.785891 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:37:22.786195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:22.787912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.800028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1321/2608] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:37:22.804900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.814427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c54521-0dfa-470b-9377-65881d3bd22e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:22.819691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:22.820362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.820654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.820944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.821668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:22.823558 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.851194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1288/2609] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:37:22 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:37:22.854401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.865555 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1b8c02-7158-440a-94eb-189ae49d7af4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:22.922753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:22.923450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.923727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.924004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.924717 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:22.926600 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.946656 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1322/2610] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:37:22 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:37:22.950816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.961944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d352fbd3-7c32-4acb-b905-c8146316c7d6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:22.966720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:22.967457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:22.967749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:22.968026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:22.968847 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:22.970519 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:22.981795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1289/2611] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:37:22.985732 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:22.995653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d31b1e-b82c-4489-80b1-88bdb23d08cf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:23.000186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e2e995-ec8e-46c7-aa96-7d7e5ec058d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:23.000894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2e995-ec8e-46c7-aa96-7d7e5ec058d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.001179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2e995-ec8e-46c7-aa96-7d7e5ec058d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.001462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e2e995-ec8e-46c7-aa96-7d7e5ec058d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.002185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2e995-ec8e-46c7-aa96-7d7e5ec058d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:23.003808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e2e995-ec8e-46c7-aa96-7d7e5ec058d5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:23.017173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1323/2612] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:37:23.018447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:23.029101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377435ea-3b27-4ff2-a471-d185c359e4a4 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:23.033678 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:23.034579 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.034864 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.035141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] PATH_INFO: `/v3/projects/7099c1547b744f52aacca683d2575ce5/users/90bb90458d3b48fd852e49b36fb76f39/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.049999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7099c1547b744f52aacca683d2575ce5, user_id=90bb90458d3b48fd852e49b36fb76f39, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:23.052690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:23.066034 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1290/2613] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:37:23 2022] PUT /identity/v3/projects/7099c1547b744f52aacca683d2575ce5/users/90bb90458d3b48fd852e49b36fb76f39/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:23.077778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.078088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.078369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.106279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] MFA Rules not processed for user `90bb90458d3b48fd852e49b36fb76f39`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:23.107943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:23.176702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1324/2614] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:23.187823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:23.189599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:23.212192 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:23.251046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f515f9-ad84-4bc3-9137-dfe2a1f7062e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:23.254745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8d5f6-488e-4889-930c-4d11c371b475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:23.256336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8d5f6-488e-4889-930c-4d11c371b475 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.256601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8d5f6-488e-4889-930c-4d11c371b475 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.256880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8d5f6-488e-4889-930c-4d11c371b475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.265680 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8d5f6-488e-4889-930c-4d11c371b475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:23.267180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8d5f6-488e-4889-930c-4d11c371b475 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:23.277198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1291/2615] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:37:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:37:23.301520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:23.321284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:23.372582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642fa04-dd52-40bb-ab54-808aa6e112a1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:23.381978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:23.382853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.383141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.383450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] PATH_INFO: `/v3/users/90bb90458d3b48fd852e49b36fb76f39` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.384652 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bb90458d3b48fd852e49b36fb76f39)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:23.386557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:23.438552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bb90458d3b48fd852e49b36fb76f39'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:23.445761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bb90458d3b48fd852e49b36fb76f39'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:23.446430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bb90458d3b48fd852e49b36fb76f39'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:23.447041 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bb90458d3b48fd852e49b36fb76f39'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:23.459102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bb90458d3b48fd852e49b36fb76f39'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:23.473540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1325/2616] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:37:23 2022] DELETE /identity/v3/users/90bb90458d3b48fd852e49b36fb76f39 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:37:23.478331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.478694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.479043 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.496799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:23.498096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:23.556802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1292/2617] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:23.566367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:23.567839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:23.635760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69ca269-1a92-4c80-8685-a6a69dc6797f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:23.639480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:23.640167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.640454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.640735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.650628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:23.652098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:23.661634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1326/2618] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:37:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:37:23.937524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:23.939241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:23.955455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d48b3-998a-4488-b502-7dc6c87f6221 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:23.966433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:23.967759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:23.967759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:23.967977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] PATH_INFO: `/v3/projects/7099c1547b744f52aacca683d2575ce5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:23.969682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7099c1547b744f52aacca683d2575ce5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:23.971642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:24.020068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7099c1547b744f52aacca683d2575ce5', 'request_id': 'req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:24.025912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7099c1547b744f52aacca683d2575ce5', 'request_id': 'req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:24.032151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7099c1547b744f52aacca683d2575ce5', 'request_id': 'req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:24.045939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] The token cache is being invalidate because project 7099c1547b744f52aacca683d2575ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:37:24.046228 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:24.046850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-9aef8a11-bc1b-4cb7-a2f1-16e7b3db7260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:37:24.048446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1293/2619] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:37:23 2022] DELETE /identity/v3/projects/7099c1547b744f52aacca683d2575ce5 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:37:30.272033 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:37:30.273593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:30.309878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:30.344902 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53904e03-a864-465f-b9a6-3bd48ae85573 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:37:30.395183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:37:30.396490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:30.396764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:30.397046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:30.398847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:30.483182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:37:30.484625 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:37:30.496283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1327/2620] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:37:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:37:34.042853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:39.436694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf01161-dbce-4880-af70-afaaec42cb7c None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:37:39.437308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf01161-dbce-4880-af70-afaaec42cb7c None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:37:39.437407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf01161-dbce-4880-af70-afaaec42cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:37:39.456139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-cbf01161-dbce-4880-af70-afaaec42cb7c None None] MFA Rules not processed for user `09befefeb2e84ae79e7437b30cb3d908`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:37:39.457363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-cbf01161-dbce-4880-af70-afaaec42cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:37:39.524135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1294/2621] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:37:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:37:40.491088 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=69 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:37:49.510790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=25 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:38:25.316667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:38:25.318098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:38:25.329552 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3562cdb8-ccfc-48bc-8c46-008128379865 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:38:25.334152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:38:25.334788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:38:25.335078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:38:25.335417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:38:25.337412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:38:25.408077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:38:25.410336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6fa147-41a0-4bbb-bd5f-559ceacdf7dd service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:38:25.420731 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1328/2622] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:38:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:38:35.415498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=28 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:38:36.904901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78c80-b68c-4613-9506-fc55c0a4dd60 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:38:36.905397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78c80-b68c-4613-9506-fc55c0a4dd60 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:38:36.905603 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe78c80-b68c-4613-9506-fc55c0a4dd60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:38:36.925136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-3fe78c80-b68c-4613-9506-fc55c0a4dd60 None None] MFA Rules not processed for user `09befefeb2e84ae79e7437b30cb3d908`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:38:36.926372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3fe78c80-b68c-4613-9506-fc55c0a4dd60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:38:36.933245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1295/2623] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:38:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:38:46.918769 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:39:03.152050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a04e81-bc26-48e0-8afa-298c2556c9df None None] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:03.152657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a04e81-bc26-48e0-8afa-298c2556c9df None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:03.152657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a04e81-bc26-48e0-8afa-298c2556c9df None None] PATH_INFO: `/` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:03.154152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1329/2624] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:39:03.163642 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa557-e4d9-4f95-ba97-9cefabc574e0 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:03.164251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa557-e4d9-4f95-ba97-9cefabc574e0 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:03.164478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aaa557-e4d9-4f95-ba97-9cefabc574e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:03.232332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-65aaa557-e4d9-4f95-ba97-9cefabc574e0 None None] MFA Rules not processed for user `fcefc528820f4d95b779c8972b46bd0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:39:03.233786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65aaa557-e4d9-4f95-ba97-9cefabc574e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:39:03.295884 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1296/2625] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:39:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:39:03.312938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a04e81-bc26-48e0-8afa-298c2556c9df None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:39:03.315295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6a04e81-bc26-48e0-8afa-298c2556c9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:39:03.330366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a04e81-bc26-48e0-8afa-298c2556c9df None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:39:03.334066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6960db-a0a3-42bf-a578-ea8ee302c4f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:39:03.334756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6960db-a0a3-42bf-a578-ea8ee302c4f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:03.335048 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6960db-a0a3-42bf-a578-ea8ee302c4f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:03.335366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6960db-a0a3-42bf-a578-ea8ee302c4f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:03.343201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6960db-a0a3-42bf-a578-ea8ee302c4f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:39:03.344699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6960db-a0a3-42bf-a578-ea8ee302c4f5 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:39:03.354522 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1330/2626] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:39:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:39:03.440637 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b9a26-a5be-4072-b4dd-7e96199649e3 None None] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:03.441018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b9a26-a5be-4072-b4dd-7e96199649e3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:03.441391 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b9a26-a5be-4072-b4dd-7e96199649e3 None None] PATH_INFO: `/` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:03.442793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1297/2627] 10.222.0.45 () {54 vars in 921 bytes} [Mon Jun 20 06:39:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 20 06:39:03.448248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08449792-9246-480e-bfb0-d1ecf38ef2bb None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:03.448541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08449792-9246-480e-bfb0-d1ecf38ef2bb None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:03.448824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08449792-9246-480e-bfb0-d1ecf38ef2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:03.486487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-08449792-9246-480e-bfb0-d1ecf38ef2bb None None] MFA Rules not processed for user `ec34108c1ae24ef699265e0be32e99cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:39:03.487643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-08449792-9246-480e-bfb0-d1ecf38ef2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:39:03.499962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1331/2628] 10.222.0.45 () {58 vars in 1018 bytes} [Mon Jun 20 06:39:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:39:03.511583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b9a26-a5be-4072-b4dd-7e96199649e3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:39:03.523457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b9a26-a5be-4072-b4dd-7e96199649e3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:39:03.527078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:39:03.527797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:03.528088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:03.529090 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:03.540000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:39:03.541451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:39:03.553802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1298/2629] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:39:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:39:13.483474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=15 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:39:13.548649 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=44 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:39:41.556167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:39:41.556628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:39:41.556770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:39:41.576444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] MFA Rules not processed for user `09befefeb2e84ae79e7437b30cb3d908`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:39:41.577644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:39:41.584712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1332/2630] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:39:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:39:51.569883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:21.356670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:21.358080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:21.370047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5a30cb-becc-485c-a3ca-6731ee84c0f9 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:21.374306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:21.374965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:21.375317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:21.375604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:21.377516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:21.445574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:21.447336 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:21.459808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1299/2631] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:24.730283 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:24.731730 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:24.747305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd842842-4d30-40ba-b422-b01ce324046d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:24.754562 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:24.755484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:24.755770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:24.756044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] PATH_INFO: `/v3/users/70323e3276ca4aaa87f55c5fb38837aa` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:24.757258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] RBAC: Authorizing `identity:delete_user(user_id=70323e3276ca4aaa87f55c5fb38837aa)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:24.759187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:24.827235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70323e3276ca4aaa87f55c5fb38837aa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:24.837117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70323e3276ca4aaa87f55c5fb38837aa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:24.837834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70323e3276ca4aaa87f55c5fb38837aa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:24.838795 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70323e3276ca4aaa87f55c5fb38837aa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:24.852836 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-61f7dda5-6770-434e-bdf1-b5f2063e01df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70323e3276ca4aaa87f55c5fb38837aa'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:24.871376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1333/2632] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:24 2022] DELETE /identity/v3/users/70323e3276ca4aaa87f55c5fb38837aa => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:25.043089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.058755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e89f62-d7e9-48be-a1af-29841b9898a3 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.118660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.119582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.119870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.120269 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] PATH_INFO: `/v3/projects/9fd70cbaefa049e299c3aabd01549d65` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.121493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fd70cbaefa049e299c3aabd01549d65)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.123467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.171160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fd70cbaefa049e299c3aabd01549d65', 'request_id': 'req-fa08043f-ec60-4d99-8423-2aecfce66567'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:25.180261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fd70cbaefa049e299c3aabd01549d65', 'request_id': 'req-fa08043f-ec60-4d99-8423-2aecfce66567'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:25.186582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fd70cbaefa049e299c3aabd01549d65', 'request_id': 'req-fa08043f-ec60-4d99-8423-2aecfce66567'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:25.201232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] The token cache is being invalidate because project 9fd70cbaefa049e299c3aabd01549d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:25.201522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:25.202157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fa08043f-ec60-4d99-8423-2aecfce66567 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:25.203833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1300/2633] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:25 2022] DELETE /identity/v3/projects/9fd70cbaefa049e299c3aabd01549d65 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:25.209849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.210218 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.210602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.228762 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:25.229958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:25.288978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1334/2634] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:25.296798 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.297701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.297969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.316005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:25.317195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:25.325130 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1301/2635] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:25.329297 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.339213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594a211-662d-4909-b4a2-f62fd18aefac None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.344288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.344945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.345279 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.345557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.346275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:25.346581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.348190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.360499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1335/2636] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:25.364721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.373918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57956db-b08f-4420-b9ee-322ddc604409 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.378517 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.379167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.379495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.379775 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.380461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.382349 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.396369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:25.415965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1302/2637] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:25.417274 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.428287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d8993a-829e-47b1-967b-49a9a45e6e89 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.483378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.484061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.484350 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.485168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.485860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.487811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.494486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:25.509906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1336/2638] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:25.513712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.524457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e31ae9-464d-4ae2-a511-3b13197bfa7f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.529012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.529666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.529951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.530245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.531009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.533139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.543771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1303/2639] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:25.548215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.558667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499288e6-6b7c-449d-9e07-c0e7905c6a29 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.563526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a345a76-6f47-476c-822d-5f9d059013d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.564205 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a345a76-6f47-476c-822d-5f9d059013d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.564490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a345a76-6f47-476c-822d-5f9d059013d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.564768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a345a76-6f47-476c-822d-5f9d059013d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.565498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a345a76-6f47-476c-822d-5f9d059013d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.567105 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a345a76-6f47-476c-822d-5f9d059013d1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.580427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1337/2640] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:25.581650 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.591716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca11324-f932-4879-a4f5-c0128b9367ad admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.597514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.598403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.598687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.598964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] PATH_INFO: `/v3/projects/0b0f379ab0914b3ea05a092b63a07849/users/893e891a2e2d416d925b9845f0f403b8/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.614320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b0f379ab0914b3ea05a092b63a07849, user_id=893e891a2e2d416d925b9845f0f403b8, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.616979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.636010 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1304/2641] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:25 2022] PUT /identity/v3/projects/0b0f379ab0914b3ea05a092b63a07849/users/893e891a2e2d416d925b9845f0f403b8/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:25.641158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.641451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.641729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.669476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] MFA Rules not processed for user `893e891a2e2d416d925b9845f0f403b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:25.670725 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:25.734565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1338/2642] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:25.738855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.800461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc31e67-6d8f-4440-9bae-98123aa3eda8 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.806340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.807081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.807596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.807885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.808861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.810763 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.843484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1305/2643] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:25.847982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.858767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc1e19-062f-4863-99df-009db2725bc6 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.916420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.917099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.917380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.917653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.918338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.920312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.940049 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1339/2644] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:25 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:25.943879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.955071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1578f4-3c3c-4595-a0e1-31aad304859c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.961006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.961677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.962023 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.962316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:25.963050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:25.964693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:25.977453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1306/2645] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:25.981531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:25.991624 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26434858-1f24-41cc-a483-0fc876a49a9d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:25.996201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:25.997172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:25.997514 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:25.997850 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] PATH_INFO: `/v3/projects/69c9e8181b834e5f9c86a59423aa8c77/users/87a501cf927644289be07c1ad6f055dc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.013736 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c9e8181b834e5f9c86a59423aa8c77, user_id=87a501cf927644289be07c1ad6f055dc, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:26.017109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:26.030700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1340/2646] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:25 2022] PUT /identity/v3/projects/69c9e8181b834e5f9c86a59423aa8c77/users/87a501cf927644289be07c1ad6f055dc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:26.034687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:26.045172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917dd518-eed3-4938-b856-a523cc6db70b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:26.106671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91806fe5-1b42-4f91-81c2-104709a2630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:26.107515 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91806fe5-1b42-4f91-81c2-104709a2630f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:26.107805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91806fe5-1b42-4f91-81c2-104709a2630f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:26.108119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91806fe5-1b42-4f91-81c2-104709a2630f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.108851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91806fe5-1b42-4f91-81c2-104709a2630f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:26.110481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91806fe5-1b42-4f91-81c2-104709a2630f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:26.121291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1307/2647] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:40:26.126013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:26.136397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d7b354-ae17-4ce7-8858-a6d77c8c2fa0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:26.141860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:26.142735 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:26.143023 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:26.143333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] PATH_INFO: `/v3/projects/69c9e8181b834e5f9c86a59423aa8c77/users/87a501cf927644289be07c1ad6f055dc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.145349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] RBAC: Authorizing `identity:create_grant(project_id=69c9e8181b834e5f9c86a59423aa8c77, user_id=87a501cf927644289be07c1ad6f055dc, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:26.148046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:26.158273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1341/2648] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:26 2022] PUT /identity/v3/projects/69c9e8181b834e5f9c86a59423aa8c77/users/87a501cf927644289be07c1ad6f055dc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:26.168845 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:26.169139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:26.169419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.197172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] MFA Rules not processed for user `87a501cf927644289be07c1ad6f055dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:26.198385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:26.217124 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:26.268012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1308/2649] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:26.278997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:26.280747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:26.302440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:26.339534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c10111-c413-4c07-afe8-0b4beceb80da admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:26.343208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264c6439-6ff8-4c68-ba54-ff7b804887a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:26.343946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c6439-6ff8-4c68-ba54-ff7b804887a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:26.344228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c6439-6ff8-4c68-ba54-ff7b804887a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:26.344502 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264c6439-6ff8-4c68-ba54-ff7b804887a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.354700 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c6439-6ff8-4c68-ba54-ff7b804887a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:26.356177 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264c6439-6ff8-4c68-ba54-ff7b804887a6 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:26.366005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1342/2650] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:26.744850 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:26.807587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0eb0f7-b6f3-458c-bc84-9581e69b812a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:26.812019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:26.812935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:26.813222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:26.813542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] PATH_INFO: `/v3/users/893e891a2e2d416d925b9845f0f403b8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.814755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] RBAC: Authorizing `identity:delete_user(user_id=893e891a2e2d416d925b9845f0f403b8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:26.818715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:26.874364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893e891a2e2d416d925b9845f0f403b8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:26.881816 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893e891a2e2d416d925b9845f0f403b8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:26.882570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893e891a2e2d416d925b9845f0f403b8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:26.883176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893e891a2e2d416d925b9845f0f403b8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:26.895176 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893e891a2e2d416d925b9845f0f403b8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:26.908498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1309/2651] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:26 2022] DELETE /identity/v3/users/893e891a2e2d416d925b9845f0f403b8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:26.913204 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:26.913655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:26.913896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:26.931939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:26.933137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:26.991749 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1343/2652] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:27.001437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:27.002821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:27.043338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:27.062670 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287728b-1015-4848-989d-f8cd41f14051 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:27.066362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:27.067015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:27.067333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:27.067614 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:27.076604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:27.078028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:27.087481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1310/2653] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:27.370857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:27.372467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:27.385578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de033b-2bca-4c5c-9a29-1235749227a5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:27.391080 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:27.391990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:27.392366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:27.392542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] PATH_INFO: `/v3/projects/0b0f379ab0914b3ea05a092b63a07849` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:27.394052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b0f379ab0914b3ea05a092b63a07849)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:27.395820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:27.444823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b0f379ab0914b3ea05a092b63a07849', 'request_id': 'req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.450695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b0f379ab0914b3ea05a092b63a07849', 'request_id': 'req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.457563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b0f379ab0914b3ea05a092b63a07849', 'request_id': 'req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.471419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] The token cache is being invalidate because project 0b0f379ab0914b3ea05a092b63a07849 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:27.471713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.472341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.473898 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1344/2654] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:27 2022] DELETE /identity/v3/projects/0b0f379ab0914b3ea05a092b63a07849 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:27.478068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:27.487765 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b369b7-226b-415e-93bc-848eae45c647 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:27.546006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:27.546920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:27.547224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:27.547533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] PATH_INFO: `/v3/users/87a501cf927644289be07c1ad6f055dc` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:27.548726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=87a501cf927644289be07c1ad6f055dc)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:27.550641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:27.598907 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87a501cf927644289be07c1ad6f055dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.606711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87a501cf927644289be07c1ad6f055dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.607582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87a501cf927644289be07c1ad6f055dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.608312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87a501cf927644289be07c1ad6f055dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.620148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-ec928e81-479f-4650-97aa-860ea73dbcaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87a501cf927644289be07c1ad6f055dc'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:27.633661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1311/2655] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:27 2022] DELETE /identity/v3/users/87a501cf927644289be07c1ad6f055dc => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:27.926479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:27.929380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:27.951153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2379cb-3be8-44ee-8986-09f79ccdc9d2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.013309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:28.031183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.032377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.032682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.032952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] PATH_INFO: `/v3/projects/69c9e8181b834e5f9c86a59423aa8c77` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.034224 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] RBAC: Authorizing `identity:delete_project(project_id=69c9e8181b834e5f9c86a59423aa8c77)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.036188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.090608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69c9e8181b834e5f9c86a59423aa8c77', 'request_id': 'req-e204127c-bdf6-40e9-90d8-409f8446034f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:28.096653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69c9e8181b834e5f9c86a59423aa8c77', 'request_id': 'req-e204127c-bdf6-40e9-90d8-409f8446034f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:28.103816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69c9e8181b834e5f9c86a59423aa8c77', 'request_id': 'req-e204127c-bdf6-40e9-90d8-409f8446034f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:28.120038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] The token cache is being invalidate because project 69c9e8181b834e5f9c86a59423aa8c77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:28.120430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:28.121159 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e204127c-bdf6-40e9-90d8-409f8446034f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:28.126249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1345/2656] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:27 2022] DELETE /identity/v3/projects/69c9e8181b834e5f9c86a59423aa8c77 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:28.128423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.128863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.129015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.152662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:28.154525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:28.220926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:28.239289 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1312/2657] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:28.241650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.241650 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.241806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.261791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:28.263044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:28.274660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1346/2658] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:28.277017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.289041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de22ea1-3796-4426-82d5-17cdda6c3b93 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.294047 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.294921 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.295418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.295860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.296794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:28.297880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.299705 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.316796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1313/2659] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:28.319087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.329770 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cbe2c-7c08-4aa9-a420-4ba3c21a1e6a None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.337669 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.338372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.338740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.339071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.339871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.342556 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.375427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1347/2660] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:28.377473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.387944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9392e-b885-4df0-9ff8-7e1f4baa7639 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.452188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.452915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.453235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.453516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.454212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.456183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.480827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1314/2661] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:28.482174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.493382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482b87b-989f-48bf-ac61-a99734991d07 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.498377 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.499045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.499366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.499657 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.500449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.502228 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.514525 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1348/2662] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:28.518585 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.529253 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae68a16f-c366-4282-9c4d-36c56db41ee7 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.538927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915d44fe-87b6-4d62-b059-eabfbf9f557d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.539221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d44fe-87b6-4d62-b059-eabfbf9f557d admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.539528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d44fe-87b6-4d62-b059-eabfbf9f557d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.539805 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915d44fe-87b6-4d62-b059-eabfbf9f557d admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.540580 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d44fe-87b6-4d62-b059-eabfbf9f557d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.542202 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915d44fe-87b6-4d62-b059-eabfbf9f557d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.557942 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1315/2663] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:40:28.559553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.570442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b28846-03d8-45a4-a945-ef9d026c78b1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.575331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.576247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.576534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.576815 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] PATH_INFO: `/v3/projects/077a2cbfcd3b4739954f749a03a068e2/users/5a47481369c04ce4add462a740129c78/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.592510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] RBAC: Authorizing `identity:create_grant(project_id=077a2cbfcd3b4739954f749a03a068e2, user_id=5a47481369c04ce4add462a740129c78, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.595801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.612860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1349/2664] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:28 2022] PUT /identity/v3/projects/077a2cbfcd3b4739954f749a03a068e2/users/5a47481369c04ce4add462a740129c78/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:28.624660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.624990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.625281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.653429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] MFA Rules not processed for user `5a47481369c04ce4add462a740129c78`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:28.654572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:28.756218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1316/2665] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:28.757479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.850262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa0ed38-6018-469e-bd73-5f5f0d3cf065 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.859150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:28.860054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:28.860443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:28.860828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:28.861622 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:28.864818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:28.900569 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:28.914086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1350/2666] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/projects => generated 381 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:28.918086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:28.929924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f3ec4d-dd6a-418b-ab3c-d4c6604eca07 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:28.998087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:29.011189 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.011911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.012198 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.012479 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.013185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.015118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.038739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1317/2667] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:28 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:29.042628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:29.057322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c87a4-a790-4dcd-9789-a350753af1ed admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:29.064784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.065469 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.065753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.066036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.066903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.069474 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.091173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1351/2668] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:40:29.095806 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:29.109941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f395cdbf-9a01-430a-822c-6d641df6933c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:29.119472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.120615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.121101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.121520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] PATH_INFO: `/v3/projects/5db7ab595f954fe481791762e497f35c/users/0f14d5ea1278473d8240ee5a9edd2b99/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.142533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db7ab595f954fe481791762e497f35c, user_id=0f14d5ea1278473d8240ee5a9edd2b99, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.145916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.163881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1318/2669] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:29 2022] PUT /identity/v3/projects/5db7ab595f954fe481791762e497f35c/users/0f14d5ea1278473d8240ee5a9edd2b99/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:29.167911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:29.179557 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc405a3d-2776-42eb-9cab-2d3e2aafed4e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:29.246628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37b6c2-6b8d-4c4a-b906-d757739d7bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.247050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37b6c2-6b8d-4c4a-b906-d757739d7bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.247479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37b6c2-6b8d-4c4a-b906-d757739d7bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.247958 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37b6c2-6b8d-4c4a-b906-d757739d7bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.248772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37b6c2-6b8d-4c4a-b906-d757739d7bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.251429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37b6c2-6b8d-4c4a-b906-d757739d7bd1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.266919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1352/2670] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:29.268373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:29.280277 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0b9468-4da5-4026-bf7a-00f831a23d72 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:29.285582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.286711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.287175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.287682 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] PATH_INFO: `/v3/projects/5db7ab595f954fe481791762e497f35c/users/0f14d5ea1278473d8240ee5a9edd2b99/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.289871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db7ab595f954fe481791762e497f35c, user_id=0f14d5ea1278473d8240ee5a9edd2b99, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.292827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.305217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1319/2671] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:29 2022] PUT /identity/v3/projects/5db7ab595f954fe481791762e497f35c/users/0f14d5ea1278473d8240ee5a9edd2b99/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:29.316564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.317169 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.318054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.362153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] MFA Rules not processed for user `0f14d5ea1278473d8240ee5a9edd2b99`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:29.363740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:29.452960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1353/2672] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:29.466525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:29.469211 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:29.600587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc36cdd-5666-4411-8cc3-c73d5645b31c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:29.605598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.606495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.606936 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.607527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.620438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.622191 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.635193 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1320/2673] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:29.741643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:29.752416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d6557-5281-4986-8f06-c256b33b01ee None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:29.756009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:29.756690 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:29.756975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:29.757248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:29.827417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:29.828903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:29.840226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1354/2674] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:32.587610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:32.653467 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610aada2-28b8-469f-b23c-ce7fdc9ec570 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:32.659081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:32.660109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:32.660497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:32.660875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] PATH_INFO: `/v3/users/5a47481369c04ce4add462a740129c78` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:32.662274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a47481369c04ce4add462a740129c78)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:32.664331 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:32.744856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a47481369c04ce4add462a740129c78'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:32.750770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:32.760739 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a47481369c04ce4add462a740129c78'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:32.761740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a47481369c04ce4add462a740129c78'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:32.762400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a47481369c04ce4add462a740129c78'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:32.794499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2b7aa39-87dd-401b-818c-5eec7003dd2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a47481369c04ce4add462a740129c78'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:32.797006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:32.798546 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:32.809800 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=100 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:32.813910 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1321/2675] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:32 2022] DELETE /identity/v3/users/5a47481369c04ce4add462a740129c78 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:32.821027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:32.821556 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:32.822074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:32.846656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:32.847969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:32.873874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2a02bd-a5d3-4475-93a4-d51505428f3d service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:32.923284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1322/2676] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:32.933042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:32.934692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:32.939688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:32.940353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:32.940643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:32.940926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:32.942693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:32.948588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f9ebd-b2b6-4b37-acda-8b72153d6fd5 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:32.954648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:32.955339 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:32.955631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:32.955906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:32.957568 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:32.959125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:32.965728 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:32.967423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:32.970378 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1355/2677] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:40:32.978615 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1323/2678] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:40:33.425582 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:33.427112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:33.441881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f24006-3b91-4cc0-80ae-9c95d0524736 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:33.447415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:33.448417 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:33.448810 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:33.449183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] PATH_INFO: `/v3/projects/077a2cbfcd3b4739954f749a03a068e2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:33.450651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=077a2cbfcd3b4739954f749a03a068e2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:33.453564 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:33.542265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '077a2cbfcd3b4739954f749a03a068e2', 'request_id': 'req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.555945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '077a2cbfcd3b4739954f749a03a068e2', 'request_id': 'req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.564733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '077a2cbfcd3b4739954f749a03a068e2', 'request_id': 'req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.579199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] The token cache is being invalidate because project 077a2cbfcd3b4739954f749a03a068e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:33.579561 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.580169 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.582117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1356/2679] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:33 2022] DELETE /identity/v3/projects/077a2cbfcd3b4739954f749a03a068e2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:33.586532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:33.598247 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6146f-8e16-4bf5-955a-e851a50ee939 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:33.665578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:33.666504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:33.666790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:33.667064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] PATH_INFO: `/v3/users/0f14d5ea1278473d8240ee5a9edd2b99` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:33.668471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f14d5ea1278473d8240ee5a9edd2b99)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:33.670388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:33.728510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f14d5ea1278473d8240ee5a9edd2b99'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.737760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f14d5ea1278473d8240ee5a9edd2b99'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.738522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f14d5ea1278473d8240ee5a9edd2b99'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.739131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f14d5ea1278473d8240ee5a9edd2b99'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.740923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:33.756749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-439e149d-aa6d-4b4d-b0b7-f799d1de0cc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f14d5ea1278473d8240ee5a9edd2b99'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:33.775803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1324/2680] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:33 2022] DELETE /identity/v3/users/0f14d5ea1278473d8240ee5a9edd2b99 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:34.133086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.134639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:34.155018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8e83a-9c99-4694-a8e7-5d34958b1a34 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.232859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:34.255182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.257199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.257494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.258024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] PATH_INFO: `/v3/projects/5db7ab595f954fe481791762e497f35c` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.259824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db7ab595f954fe481791762e497f35c)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.261785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.328409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db7ab595f954fe481791762e497f35c', 'request_id': 'req-8f632419-7a49-4a6e-8549-be8494ca3080'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:34.334955 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db7ab595f954fe481791762e497f35c', 'request_id': 'req-8f632419-7a49-4a6e-8549-be8494ca3080'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:34.341903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db7ab595f954fe481791762e497f35c', 'request_id': 'req-8f632419-7a49-4a6e-8549-be8494ca3080'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:34.358630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] The token cache is being invalidate because project 5db7ab595f954fe481791762e497f35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:34.358929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:34.359727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8f632419-7a49-4a6e-8549-be8494ca3080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:34.364883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1357/2681] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:34 2022] DELETE /identity/v3/projects/5db7ab595f954fe481791762e497f35c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:34.366704 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.367508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.368013 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.398959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:34.408666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:34.503371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1325/2682] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:40:34.554485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.554778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.555046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.577082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:34.579641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:34.590761 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1358/2683] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:34.600383 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.610329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05727258-0d67-4d5e-b02f-23a628cf7d30 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.615071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.616770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.619334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.619685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.620620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:34.620943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.622553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.638827 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1326/2684] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:34.641073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.650990 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53382659-01f6-42cf-9f71-62eb204d3b22 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.656059 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.656966 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.657079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.657286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.657985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.659943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.691680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1359/2685] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:34 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:34.695781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.710142 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8876161-dfe9-4695-9572-8df6631ca370 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.782628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.785019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.785352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.785601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.786324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.788366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.822989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1327/2686] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:34 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:34.826992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.838290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece3002-e437-4ca9-bd7b-abcc9c5deb6e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.843630 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.844513 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.844977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.845558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.846478 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.848416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.861385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1360/2687] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:34.865118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.878470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4501b7-8784-4948-bbcb-3699473c7384 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.884833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd34d1f-c547-4b11-8e49-5eaced35a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.885966 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34d1f-c547-4b11-8e49-5eaced35a2df admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.886444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34d1f-c547-4b11-8e49-5eaced35a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.886981 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd34d1f-c547-4b11-8e49-5eaced35a2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.887996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd34d1f-c547-4b11-8e49-5eaced35a2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.889882 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd34d1f-c547-4b11-8e49-5eaced35a2df admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.901951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1328/2688] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:34.906381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:34.917000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677daa05-df07-4c19-b9ef-af13a91684eb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:34.922704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:34.923646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.923925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.924314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] PATH_INFO: `/v3/projects/9681a31a25e5416ca26bfc7b0c648aac/users/f001d069bc4749a5b0b96f4e539aa536/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:34.944245 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9681a31a25e5416ca26bfc7b0c648aac, user_id=f001d069bc4749a5b0b96f4e539aa536, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:34.947977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:34.967284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1361/2689] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:34 2022] PUT /identity/v3/projects/9681a31a25e5416ca26bfc7b0c648aac/users/f001d069bc4749a5b0b96f4e539aa536/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:34.977209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:34.977506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:34.977790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.012695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] MFA Rules not processed for user `f001d069bc4749a5b0b96f4e539aa536`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:35.013588 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:35.049713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:35.108824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1329/2690] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:35.110626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.182060 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677a3e3-0de5-4983-8bef-95c90502b7fd admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.190206 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.191236 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.191738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.192262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.193330 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.195662 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.222908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:35.236949 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1362/2691] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:35 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:35.237951 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.254187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ff93-6ecb-4357-9648-220c54a22473 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.332260 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.333372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.333891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.334371 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.335313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.337644 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.365909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1330/2692] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:35 2022] POST /identity/v3/users => generated 301 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:35.369733 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.382453 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d075c06-5707-44e3-988d-f3b0faf7ea2d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.387998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.388691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.389152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.389444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.390713 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.392978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.414150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1363/2693] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:35.415233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.428101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874df4b0-f819-40b9-a9b4-1918fba93220 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.433236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.434141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.434425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.434701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] PATH_INFO: `/v3/projects/54ce8dbf7dd44733b262a7e541964f93/users/5dd2409b71864a2fafbd3ff37ffd6feb/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.454226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ce8dbf7dd44733b262a7e541964f93, user_id=5dd2409b71864a2fafbd3ff37ffd6feb, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.456985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.478096 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1331/2694] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:35 2022] PUT /identity/v3/projects/54ce8dbf7dd44733b262a7e541964f93/users/5dd2409b71864a2fafbd3ff37ffd6feb/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:35.479679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.491460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05db4f4-23a2-45bd-92c1-2d80c08f2ec5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.576376 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a184a2b-9cc5-4656-a7e5-964a92a13f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.577093 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184a2b-9cc5-4656-a7e5-964a92a13f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.577382 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184a2b-9cc5-4656-a7e5-964a92a13f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.578349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184a2b-9cc5-4656-a7e5-964a92a13f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.579145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a184a2b-9cc5-4656-a7e5-964a92a13f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.580821 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a184a2b-9cc5-4656-a7e5-964a92a13f48 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.594992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1364/2695] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:35.596187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.610471 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf435f-55f3-4c15-9cba-1baac45f1c96 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.618685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.620106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.620455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.621197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] PATH_INFO: `/v3/projects/54ce8dbf7dd44733b262a7e541964f93/users/5dd2409b71864a2fafbd3ff37ffd6feb/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.623630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] RBAC: Authorizing `identity:create_grant(project_id=54ce8dbf7dd44733b262a7e541964f93, user_id=5dd2409b71864a2fafbd3ff37ffd6feb, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.626766 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.642324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1332/2696] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:35 2022] PUT /identity/v3/projects/54ce8dbf7dd44733b262a7e541964f93/users/5dd2409b71864a2fafbd3ff37ffd6feb/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:35.652585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.652885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.653173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.686468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] MFA Rules not processed for user `5dd2409b71864a2fafbd3ff37ffd6feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:35.687746 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:35.768220 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1365/2697] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:35.780395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:35.784846 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:35.861781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b5e64-b8aa-49ae-ae45-6c3274a6aabc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:35.877880 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79752cd6-e211-476f-aa52-eda7b257b614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:35.878743 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752cd6-e211-476f-aa52-eda7b257b614 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:35.879037 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752cd6-e211-476f-aa52-eda7b257b614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:35.883401 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79752cd6-e211-476f-aa52-eda7b257b614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:35.895922 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79752cd6-e211-476f-aa52-eda7b257b614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:35.897786 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79752cd6-e211-476f-aa52-eda7b257b614 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:35.911830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1333/2698] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:36.073753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:36.138704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa5403-f8fc-4057-87b3-6011ced84684 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:36.143840 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:36.144799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:36.145161 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:36.145516 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] PATH_INFO: `/v3/users/f001d069bc4749a5b0b96f4e539aa536` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:36.146849 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f001d069bc4749a5b0b96f4e539aa536)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:36.149037 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:36.162570 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:36.204430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f001d069bc4749a5b0b96f4e539aa536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:36.212429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f001d069bc4749a5b0b96f4e539aa536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:36.217085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f001d069bc4749a5b0b96f4e539aa536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:36.217085 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f001d069bc4749a5b0b96f4e539aa536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:36.233414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f001d069bc4749a5b0b96f4e539aa536'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:36.249308 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1366/2699] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:36 2022] DELETE /identity/v3/users/f001d069bc4749a5b0b96f4e539aa536 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:36.254522 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:36.254984 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:36.255604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:36.277660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:36.279000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:36.303012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:36.454833 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1334/2700] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:36.464759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:36.466704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:36.605441 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9e3a5d-e892-4af6-a6df-f601e1ea55a7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:36.616150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:36.616455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:36.616702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:36.617014 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:36.633765 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:36.636341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:36.663026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1367/2701] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:37.082068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:37.084516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:37.098627 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60575b4-6006-459d-91e7-8758181c2576 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:37.104089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:37.105049 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:37.105413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:37.105789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] PATH_INFO: `/v3/projects/9681a31a25e5416ca26bfc7b0c648aac` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:37.107077 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9681a31a25e5416ca26bfc7b0c648aac)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:37.109131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:37.181930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9681a31a25e5416ca26bfc7b0c648aac', 'request_id': 'req-6071324c-b905-4a7a-b8ed-f1ec694168aa'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.194080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9681a31a25e5416ca26bfc7b0c648aac', 'request_id': 'req-6071324c-b905-4a7a-b8ed-f1ec694168aa'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.201335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9681a31a25e5416ca26bfc7b0c648aac', 'request_id': 'req-6071324c-b905-4a7a-b8ed-f1ec694168aa'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.216412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] The token cache is being invalidate because project 9681a31a25e5416ca26bfc7b0c648aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:37.216708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.217314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.219794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1335/2702] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:37 2022] DELETE /identity/v3/projects/9681a31a25e5416ca26bfc7b0c648aac => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:37.223829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:37.234415 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfe915e-cfec-4708-827d-7359fd2f93bb service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:37.297776 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:37.298687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:37.298975 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:37.299275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] PATH_INFO: `/v3/users/5dd2409b71864a2fafbd3ff37ffd6feb` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:37.300468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5dd2409b71864a2fafbd3ff37ffd6feb)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:37.302423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:37.344774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:37.358547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5dd2409b71864a2fafbd3ff37ffd6feb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.366179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5dd2409b71864a2fafbd3ff37ffd6feb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.366882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5dd2409b71864a2fafbd3ff37ffd6feb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.367590 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5dd2409b71864a2fafbd3ff37ffd6feb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.381082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cc104145-7dfb-42fa-bd5e-534b5f1eff28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5dd2409b71864a2fafbd3ff37ffd6feb'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.399036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1368/2703] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:37 2022] DELETE /identity/v3/users/5dd2409b71864a2fafbd3ff37ffd6feb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:37.711006 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:37.712484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:37.728771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6071324c-b905-4a7a-b8ed-f1ec694168aa admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:37.854524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:37.863909 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:37.864201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:37.864482 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] PATH_INFO: `/v3/projects/54ce8dbf7dd44733b262a7e541964f93` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:37.868912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54ce8dbf7dd44733b262a7e541964f93)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:37.874089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:37.931708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54ce8dbf7dd44733b262a7e541964f93', 'request_id': 'req-6ba9b493-893b-4c07-8a1c-9a0be8657680'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.933222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:37.940807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54ce8dbf7dd44733b262a7e541964f93', 'request_id': 'req-6ba9b493-893b-4c07-8a1c-9a0be8657680'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.949886 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54ce8dbf7dd44733b262a7e541964f93', 'request_id': 'req-6ba9b493-893b-4c07-8a1c-9a0be8657680'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.966110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] The token cache is being invalidate because project 54ce8dbf7dd44733b262a7e541964f93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:37.966110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.966977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6ba9b493-893b-4c07-8a1c-9a0be8657680 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:37.972648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1336/2704] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:37 2022] DELETE /identity/v3/projects/54ce8dbf7dd44733b262a7e541964f93 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:37.974691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:37.975306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:37.975602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:37.996374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:37.997615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:38.131545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1369/2705] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:38.141537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.142074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.142155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.181083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:38.182489 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:38.201285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1337/2706] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:38.202980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.216991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48c024-9e50-477f-9428-5edb8d4d329d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.225094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.226579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.226964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.227475 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.228314 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:38.228723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.231670 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:38.250247 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1370/2707] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:38.254715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.264195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c01374-339d-42b3-8682-d8ef0e6e0a4f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.273830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.274686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.275126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.275594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.277068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.280454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:38.314452 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1338/2708] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:38 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:38.318237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.329910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733538c-ceb3-4b07-8f7e-bd7ae5b13372 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.434246 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.435374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.435876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.436396 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.437510 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.439824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:38.467825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1371/2709] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:38 2022] POST /identity/v3/users => generated 290 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:38.472656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.485204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad44301-c89c-463a-a49c-3ea079f0db8a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.492538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.493433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.493994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.494455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.495408 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.497227 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:38.514760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1339/2710] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:38.515888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.526951 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9bb2a2-1801-43c2-827b-5fd153a23f08 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.532709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442432bd-3397-49e6-bc3b-2badf14834cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.533560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442432bd-3397-49e6-bc3b-2badf14834cb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.533856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442432bd-3397-49e6-bc3b-2badf14834cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.534145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442432bd-3397-49e6-bc3b-2badf14834cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.534878 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442432bd-3397-49e6-bc3b-2badf14834cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.536553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442432bd-3397-49e6-bc3b-2badf14834cb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:38.574397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1372/2711] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:40:38.575959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.592370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4f9d9-5818-4094-9148-69d1528095a5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.601634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.602541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.602828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.603111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] PATH_INFO: `/v3/projects/6cf88eec5d0944ad82f0e714ecfb7100/users/8adf96ddd0384fc59fe9fed2ee953a6a/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.633248 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cf88eec5d0944ad82f0e714ecfb7100, user_id=8adf96ddd0384fc59fe9fed2ee953a6a, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.637592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:38.672363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1340/2712] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:38 2022] PUT /identity/v3/projects/6cf88eec5d0944ad82f0e714ecfb7100/users/8adf96ddd0384fc59fe9fed2ee953a6a/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:38.673512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.673512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.673692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.711613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] MFA Rules not processed for user `8adf96ddd0384fc59fe9fed2ee953a6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:38.713693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:38.720153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:38.799455 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1373/2713] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:38.800959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:38.917387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfc764-a444-43f1-97a9-0add3925b370 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:38.931445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:38.932240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:38.935285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:38.935583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:38.937697 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:38.941871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.026406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1341/2714] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:38 2022] POST /identity/v3/projects => generated 379 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:39.027468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:39.044015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae57a4-81c4-4567-962e-bca01508d846 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:39.170663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:39.172740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.172740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.173003 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.173831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:39.176668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.212752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:39.214985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1374/2715] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:39 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:39.225460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fdbe0e-8cae-4b60-9b13-3ab77b9d2bd6 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:39.232218 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:39.233291 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.234610 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.235066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.236066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:39.240204 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.259964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:39.261020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1342/2716] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:39.282270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b8648-4f00-43ed-87d8-578143f0115b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:39.287859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:39.289021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.289363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.290197 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] PATH_INFO: `/v3/projects/31aa8a1b807549e387b51fff5429fe6d/users/3f338faea50e46f0b074630b4b609754/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.315422 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31aa8a1b807549e387b51fff5429fe6d, user_id=3f338faea50e46f0b074630b4b609754, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:39.317870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.355487 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:39.367011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1375/2717] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:39 2022] PUT /identity/v3/projects/31aa8a1b807549e387b51fff5429fe6d/users/3f338faea50e46f0b074630b4b609754/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:39.368841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e242bfb-8f22-4ef1-9eff-8ae7893858b9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:39.425154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:39.442369 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc22732-3548-4295-b2fd-9e33afd49f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:39.443135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22732-3548-4295-b2fd-9e33afd49f97 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.443481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22732-3548-4295-b2fd-9e33afd49f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.443750 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc22732-3548-4295-b2fd-9e33afd49f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.444564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22732-3548-4295-b2fd-9e33afd49f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:39.446822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc22732-3548-4295-b2fd-9e33afd49f97 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.463767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:39.473906 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d83ea-adbf-4207-b3ae-e133f5581835 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:39.474863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1343/2718] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:39 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:39.479066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:39.479978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.480798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.480798 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] PATH_INFO: `/v3/projects/31aa8a1b807549e387b51fff5429fe6d/users/3f338faea50e46f0b074630b4b609754/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.482566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=31aa8a1b807549e387b51fff5429fe6d, user_id=3f338faea50e46f0b074630b4b609754, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:39.485728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.499439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1376/2719] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:39 2022] PUT /identity/v3/projects/31aa8a1b807549e387b51fff5429fe6d/users/3f338faea50e46f0b074630b4b609754/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:39.510929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.511360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.511564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.550543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] MFA Rules not processed for user `3f338faea50e46f0b074630b4b609754`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:39.552990 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:39.692941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1344/2720] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:39.706097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:39.708104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:39.792044 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3959b944-d9c2-4d42-b2e7-125140a653cc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:39.799243 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8585efa1-17d7-4202-9f10-d5f6ea701644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:39.800187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585efa1-17d7-4202-9f10-d5f6ea701644 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:39.800673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585efa1-17d7-4202-9f10-d5f6ea701644 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:39.801410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8585efa1-17d7-4202-9f10-d5f6ea701644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:39.812856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585efa1-17d7-4202-9f10-d5f6ea701644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:39.814450 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8585efa1-17d7-4202-9f10-d5f6ea701644 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:39.835472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1377/2721] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:39 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:40.341669 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:40.504796 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df34655f-68ce-4c2b-8aaf-9d2e25c2f7e4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:40.519318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:40.520601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:40.521457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:40.522355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] PATH_INFO: `/v3/users/8adf96ddd0384fc59fe9fed2ee953a6a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:40.523964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8adf96ddd0384fc59fe9fed2ee953a6a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:40.527110 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:40.618740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8adf96ddd0384fc59fe9fed2ee953a6a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:40.627716 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8adf96ddd0384fc59fe9fed2ee953a6a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:40.628470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8adf96ddd0384fc59fe9fed2ee953a6a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:40.629106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8adf96ddd0384fc59fe9fed2ee953a6a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:40.678756 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:40.681022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8adf96ddd0384fc59fe9fed2ee953a6a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:40.708802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1345/2722] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:40 2022] DELETE /identity/v3/users/8adf96ddd0384fc59fe9fed2ee953a6a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:40.710443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:40.710953 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:40.711045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:40.717548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:40.746944 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:40.748213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:40.903737 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1378/2723] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:40.904855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:40.908011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:41.005789 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346635e4-702b-474f-81ee-de44052db45b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:41.012898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:41.013712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:41.014055 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:41.015740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:41.029566 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:41.034311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:41.060734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:41.062282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:41.071578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f8fa94-d44e-4c0d-a30d-ab2dfc960c21 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:41.078505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:41.079540 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:41.079879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:41.080168 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] PATH_INFO: `/v3/users/4c005728531546aabec0720d9bb1c636` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:41.081563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c005728531546aabec0720d9bb1c636)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:41.084527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:41.097307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1346/2724] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:40:41.162158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c005728531546aabec0720d9bb1c636'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.175593 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c005728531546aabec0720d9bb1c636'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.176718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c005728531546aabec0720d9bb1c636'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.178380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c005728531546aabec0720d9bb1c636'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.209825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c005728531546aabec0720d9bb1c636'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.227644 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1379/2725] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:41 2022] DELETE /identity/v3/users/4c005728531546aabec0720d9bb1c636 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:41.502207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:41.505994 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:41.522889 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00427c7-04a2-4b2f-8be1-1681f41f59e9 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:41.539192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:41.541047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:41.605276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:41.606152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:41.606428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:41.606707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] PATH_INFO: `/v3/projects/6cf88eec5d0944ad82f0e714ecfb7100` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:41.607938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cf88eec5d0944ad82f0e714ecfb7100)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:41.609923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:41.632013 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426b509-c646-4e7d-847b-fe25fe466b6e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:41.637234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:41.638038 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:41.638317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:41.638833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] PATH_INFO: `/v3/projects/aee5c8d850d143cbac59ea9fb688ac68` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:41.640790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee5c8d850d143cbac59ea9fb688ac68)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:41.643400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:41.659126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cf88eec5d0944ad82f0e714ecfb7100', 'request_id': 'req-e3a20072-7689-4893-899c-84dabe42b446'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.663186 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:41.665630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cf88eec5d0944ad82f0e714ecfb7100', 'request_id': 'req-e3a20072-7689-4893-899c-84dabe42b446'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.674446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cf88eec5d0944ad82f0e714ecfb7100', 'request_id': 'req-e3a20072-7689-4893-899c-84dabe42b446'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.691153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] The token cache is being invalidate because project 6cf88eec5d0944ad82f0e714ecfb7100 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:41.691478 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.692129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.693841 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1347/2726] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:41 2022] DELETE /identity/v3/projects/6cf88eec5d0944ad82f0e714ecfb7100 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:41.697848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:41.699226 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:41.703485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee5c8d850d143cbac59ea9fb688ac68', 'request_id': 'req-c437800d-69df-4538-aef1-db31383566ed'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.709680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee5c8d850d143cbac59ea9fb688ac68', 'request_id': 'req-c437800d-69df-4538-aef1-db31383566ed'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.716901 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee5c8d850d143cbac59ea9fb688ac68', 'request_id': 'req-c437800d-69df-4538-aef1-db31383566ed'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.728457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:41.732261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] The token cache is being invalidate because project aee5c8d850d143cbac59ea9fb688ac68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:41.732479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.733349 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.735578 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1380/2727] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:41 2022] DELETE /identity/v3/projects/aee5c8d850d143cbac59ea9fb688ac68 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:41.739117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:41.741392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:41.775761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a20072-7689-4893-899c-84dabe42b446 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:41.809445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437800d-69df-4538-aef1-db31383566ed admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:41.815116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:41.821167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:41.821723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:41.822087 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] PATH_INFO: `/v3/users/8a98b2c69e1e49dfa7858b8616a538d8` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:41.823888 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a98b2c69e1e49dfa7858b8616a538d8)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:41.827004 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:41.844203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:41.845237 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:41.845629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:41.846029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] PATH_INFO: `/v3/users/3f338faea50e46f0b074630b4b609754` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:41.847403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f338faea50e46f0b074630b4b609754)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:41.849815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:41.885663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a98b2c69e1e49dfa7858b8616a538d8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.894141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a98b2c69e1e49dfa7858b8616a538d8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.894881 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a98b2c69e1e49dfa7858b8616a538d8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.895530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a98b2c69e1e49dfa7858b8616a538d8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.907169 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f338faea50e46f0b074630b4b609754'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.909117 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a98b2c69e1e49dfa7858b8616a538d8'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.915543 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f338faea50e46f0b074630b4b609754'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.916437 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f338faea50e46f0b074630b4b609754'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.917138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f338faea50e46f0b074630b4b609754'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.923233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1381/2728] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:41 2022] DELETE /identity/v3/users/8a98b2c69e1e49dfa7858b8616a538d8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:41.929701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f338faea50e46f0b074630b4b609754'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:41.943154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1348/2729] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:41 2022] DELETE /identity/v3/users/3f338faea50e46f0b074630b4b609754 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:42.116654 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.118298 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:42.158425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:42.212986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724c92-a576-4071-9a9a-653a66f0df6d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.266251 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.267944 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:42.278467 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.279443 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.279738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.280024 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] PATH_INFO: `/v3/projects/8ce83c74885b429eb85d9bbd65e53c5f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.282251 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce83c74885b429eb85d9bbd65e53c5f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.284908 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.353409 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:42.357134 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce83c74885b429eb85d9bbd65e53c5f', 'request_id': 'req-5e483e38-c290-4b54-b8ae-d9285c43d96e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.367124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce83c74885b429eb85d9bbd65e53c5f', 'request_id': 'req-5e483e38-c290-4b54-b8ae-d9285c43d96e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.380633 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce83c74885b429eb85d9bbd65e53c5f', 'request_id': 'req-5e483e38-c290-4b54-b8ae-d9285c43d96e'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.389020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e904bf76-eb42-4375-a22c-a8e1c684422c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.402505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] The token cache is being invalidate because project 8ce83c74885b429eb85d9bbd65e53c5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:42.402839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.404772 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-5e483e38-c290-4b54-b8ae-d9285c43d96e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.408876 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1382/2730] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:42 2022] DELETE /identity/v3/projects/8ce83c74885b429eb85d9bbd65e53c5f => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:42.413335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a38a72-c61c-4ce2-9d08-acc8cff74df2 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.413631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a38a72-c61c-4ce2-9d08-acc8cff74df2 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.413919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a38a72-c61c-4ce2-9d08-acc8cff74df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.436466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-73a38a72-c61c-4ce2-9d08-acc8cff74df2 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:42.438355 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-73a38a72-c61c-4ce2-9d08-acc8cff74df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:42.494022 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.494896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.495180 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.495493 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] PATH_INFO: `/v3/projects/31aa8a1b807549e387b51fff5429fe6d` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.496712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31aa8a1b807549e387b51fff5429fe6d)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.498813 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.523920 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1383/2731] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:42.529560 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.529969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.530385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.538751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:42.560871 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:42.562235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:42.562553 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31aa8a1b807549e387b51fff5429fe6d', 'request_id': 'req-34edcf5d-109d-40a1-adb9-79bbd0488df5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.569413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31aa8a1b807549e387b51fff5429fe6d', 'request_id': 'req-34edcf5d-109d-40a1-adb9-79bbd0488df5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.575932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31aa8a1b807549e387b51fff5429fe6d', 'request_id': 'req-34edcf5d-109d-40a1-adb9-79bbd0488df5'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.590436 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] The token cache is being invalidate because project 31aa8a1b807549e387b51fff5429fe6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:42.591446 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.592195 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-34edcf5d-109d-40a1-adb9-79bbd0488df5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:42.595404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1349/2732] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:42 2022] DELETE /identity/v3/projects/31aa8a1b807549e387b51fff5429fe6d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:42.600264 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e06f84-35ba-4774-b77e-de113471a7cc None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.600629 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e06f84-35ba-4774-b77e-de113471a7cc None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.600985 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e06f84-35ba-4774-b77e-de113471a7cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.620520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d6e06f84-35ba-4774-b77e-de113471a7cc None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:42.621867 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d6e06f84-35ba-4774-b77e-de113471a7cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:42.638833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1384/2733] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:40:42.639702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.649067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7eaeb-0427-4007-91ce-ed81e2bd674f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.654143 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.654916 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.655335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.656271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.657135 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:42.657526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.659226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.677010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1385/2734] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:42.678344 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.691217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1350/2735] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:42.699142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ca7a73-8a91-4693-a0d6-2fd7e95a360f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.699324 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.699559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.699834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.706978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.707685 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.707978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.708260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.709322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.711580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.719917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:42.721327 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:42.733689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1351/2736] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:42.735513 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.746139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d61b9e-bde5-4e7d-8866-1eef8a32850e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.751172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.752009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.752246 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.752546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.753304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:42.753834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.755428 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.757807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1386/2737] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:42.761870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.773031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1352/2738] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:42.774256 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.785941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27ab7e-fd56-4fa9-b783-e1f621ffdf3c admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.790431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:42.864429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.865925 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.866330 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.866722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.867738 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.869054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f420f75-e42e-42bc-810a-273a45543e09 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.870971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.875054 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.876258 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.876862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.877311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.878594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.881190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.909371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1387/2739] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/users => generated 290 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:42.913244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.918801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1353/2740] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:42.920634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:42.924187 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562318ed-5721-43c1-bbf3-231b1c43b8d9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:42.979899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:42.988529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:42.989538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:42.989875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:42.990162 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:42.991151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:42.994097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:42.998413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9449cd-68ac-460d-8118-8ad7b20dfc6b admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.004185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.005074 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.005594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.006044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.006938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.007854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1388/2741] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:40:43.008793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.009898 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.019788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66483b4-14b0-476e-b0d1-ec6485499e01 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.025994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.026804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.027428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.027860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.029025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.031667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.043823 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1354/2742] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:42 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:43.046899 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.047399 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1389/2743] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:43.048409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.058985 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1cd9f4-14fc-4717-927b-9236f7c4f865 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.064341 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe38cd-ea9e-4b36-8134-03cc5aec11d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.064770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7669081f-613d-46c9-a9ff-ef9adbdd210e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.065494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe38cd-ea9e-4b36-8134-03cc5aec11d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.065883 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe38cd-ea9e-4b36-8134-03cc5aec11d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.066265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe38cd-ea9e-4b36-8134-03cc5aec11d4 admin admin] PATH_INFO: `/v3/projects/6a9945a382d749c99a76913dcce97740/users/0c0fdef83aed4170b9d0db550daa17d0/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.071895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.072572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.072999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.073429 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.074206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.076849 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.084094 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe38cd-ea9e-4b36-8134-03cc5aec11d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a9945a382d749c99a76913dcce97740, user_id=0c0fdef83aed4170b9d0db550daa17d0, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.086940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe38cd-ea9e-4b36-8134-03cc5aec11d4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.093972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1355/2744] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:43.095038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.104291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1390/2745] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:43 2022] PUT /identity/v3/projects/6a9945a382d749c99a76913dcce97740/users/0c0fdef83aed4170b9d0db550daa17d0/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:43.107819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad0461-a081-4e30-b7d1-5baf3129c039 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.109860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.110262 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.110728 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.129875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:43.131380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:43.177938 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.178804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.179100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.179419 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.180322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.182002 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.193676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1356/2746] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:43.198137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.210274 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2178b-2aa7-46e8-bd7a-a28e7ede04ed admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.211880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1391/2747] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:43.213070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.214954 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:43.215548 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f8de3-4821-43ae-8500-dab90355fb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.216464 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8de3-4821-43ae-8500-dab90355fb10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.216749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8de3-4821-43ae-8500-dab90355fb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.217035 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f8de3-4821-43ae-8500-dab90355fb10 admin admin] PATH_INFO: `/v3/projects/e16441a7b0ca47318470f9e77f388f08/users/557f432d75114d028b0c81c43a44e38f/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.220856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:43.236578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8de3-4821-43ae-8500-dab90355fb10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e16441a7b0ca47318470f9e77f388f08, user_id=557f432d75114d028b0c81c43a44e38f, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.239598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f8de3-4821-43ae-8500-dab90355fb10 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.270727 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1357/2748] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:43 2022] PUT /identity/v3/projects/e16441a7b0ca47318470f9e77f388f08/users/557f432d75114d028b0c81c43a44e38f/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:43.272979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.273275 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.273557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.285769 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4ceca1-b4a4-4889-af15-970af3589c85 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.308498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] MFA Rules not processed for user `557f432d75114d028b0c81c43a44e38f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:43.309690 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:43.341286 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.342335 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.342894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.343454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.376312 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:43.406333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1358/2749] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:43.413497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.418781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.420497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.429050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c5c25-21c2-4f1f-9545-7d8243f4b64d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.433196 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1392/2750] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:43.435531 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.436229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.436526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.437352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.438119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.440236 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.487470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1359/2751] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:43 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:40:43.491504 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.502940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623ce2ba-c866-48e1-8249-866a851a4caf service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.608227 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.610372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.611458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.611939 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.612932 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.615805 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.645541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1393/2752] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:43 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:43.646782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.658768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836f14bc-972a-4318-9d89-6938a2df28ef admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.667222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.668005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.668355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.668689 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.669495 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.671364 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.691126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1360/2753] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:43.691752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.703040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557d0dc8-eab7-4a81-8e35-f815ae52b6d8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.708309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e47cd3-f9ce-4344-8642-49c8ce9c123a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.709017 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e47cd3-f9ce-4344-8642-49c8ce9c123a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.709311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e47cd3-f9ce-4344-8642-49c8ce9c123a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.709594 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e47cd3-f9ce-4344-8642-49c8ce9c123a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.710358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e47cd3-f9ce-4344-8642-49c8ce9c123a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.712016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e47cd3-f9ce-4344-8642-49c8ce9c123a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.723301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1394/2754] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:43.728511 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:43.741387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18223c7a-6bc0-4038-8ec3-4207324d533f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:43.751069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:43.751509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.751858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.753306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] PATH_INFO: `/v3/projects/e23af21a25f741fa8871a19cd3239b09/users/ee1ffd6fb7074629b207c795d22537a0/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.782064 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e23af21a25f741fa8871a19cd3239b09, user_id=ee1ffd6fb7074629b207c795d22537a0, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:43.784803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:43.845166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1361/2755] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:43 2022] PUT /identity/v3/projects/e23af21a25f741fa8871a19cd3239b09/users/ee1ffd6fb7074629b207c795d22537a0/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:43.857663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:43.858149 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:43.858290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:43.897571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] MFA Rules not processed for user `ee1ffd6fb7074629b207c795d22537a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:43.899021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:43.998697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1395/2756] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:44.000808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:44.127338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37c38f-3cf1-45c2-bfe9-745c636cc650 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:44.138708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:44.139687 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.143297 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.143616 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.144355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:44.148908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:44.217634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:44.234790 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1362/2757] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:43 2022] POST /identity/v3/projects => generated 377 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:44.238712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:44.257238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7459727a-7714-46ce-a718-afd7d1d16b32 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:44.346720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:44.415752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:44.416539 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.416835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.417790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.418666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:44.421608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:44.466424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1396/2758] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:44 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:44.470533 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:44.489634 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e9eb11-8798-47c3-9438-4118c91f2072 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:44.495897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:44.497406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.497406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.497406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.505118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:44.507528 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:44.537667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1363/2759] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:44.542385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:44.556032 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb304c19-f537-4433-b5ff-c15e3bccba4b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:44.561388 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:44.562760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.563043 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.563963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] PATH_INFO: `/v3/projects/5394a542602d47ba9c661e4851154b28/users/73c076e22520438c97e4660a629d0c58/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.585217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5394a542602d47ba9c661e4851154b28, user_id=73c076e22520438c97e4660a629d0c58, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:44.588915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:44.615843 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1397/2760] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:44 2022] PUT /identity/v3/projects/5394a542602d47ba9c661e4851154b28/users/73c076e22520438c97e4660a629d0c58/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:44.620314 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:44.634290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94508a25-d778-4928-aee1-66ad934e3cd2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:44.751821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfd21f-dcac-4e41-91c1-8963989d11e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:44.753241 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfd21f-dcac-4e41-91c1-8963989d11e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.753536 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfd21f-dcac-4e41-91c1-8963989d11e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.753821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfd21f-dcac-4e41-91c1-8963989d11e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.754648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfd21f-dcac-4e41-91c1-8963989d11e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:44.756288 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfd21f-dcac-4e41-91c1-8963989d11e2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:44.768001 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1364/2761] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:40:44.773237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:44.785828 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770eba42-b68d-457e-828b-c599addbd5b7 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:44.792025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:44.793346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.793638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.793995 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] PATH_INFO: `/v3/projects/5394a542602d47ba9c661e4851154b28/users/73c076e22520438c97e4660a629d0c58/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.804956 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5394a542602d47ba9c661e4851154b28, user_id=73c076e22520438c97e4660a629d0c58, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:44.808783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:44.834133 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:44.834640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1398/2762] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:44 2022] PUT /identity/v3/projects/5394a542602d47ba9c661e4851154b28/users/73c076e22520438c97e4660a629d0c58/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:44.834920 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:44.835335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:44.911702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] MFA Rules not processed for user `73c076e22520438c97e4660a629d0c58`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:44.913094 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:45.092632 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1365/2763] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 20 06:40:45.110057 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:45.111959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:45.223175 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725844de-60a1-4bfe-9ac5-42fc013108e1 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:45.228694 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:45.229488 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:45.229894 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:45.237835 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:45.328459 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:45.329882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:45.341784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1399/2764] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:47.247148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:47.261015 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d5eca-8a55-495b-b9a8-108178e57a41 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:47.275170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf0284-ff47-44be-b73b-63aa05df9275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:47.276335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf0284-ff47-44be-b73b-63aa05df9275 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:47.276924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf0284-ff47-44be-b73b-63aa05df9275 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:47.277411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf0284-ff47-44be-b73b-63aa05df9275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:47.380538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf0284-ff47-44be-b73b-63aa05df9275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:47.382448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf0284-ff47-44be-b73b-63aa05df9275 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:47.411602 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1366/2765] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:48.318865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:48.328543 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e488d-c970-4e25-b61e-88093d8f9eca service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:48.332866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:48.333886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:48.334278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:48.334660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:48.347242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:48.348946 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:48.376416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1400/2766] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:49.616564 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:49.617014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:49.617322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:49.640858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:49.654578 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] MFA Rules not processed for user `0c0fdef83aed4170b9d0db550daa17d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:49.656290 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:49.757537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1367/2767] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:49.761942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:49.831531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:49.886424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81638c33-f8d3-4f03-b752-1ed54238ec6c service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:49.895189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:49.896451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:49.897265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:49.897833 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:49.898880 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:49.902910 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:49.958319 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1401/2768] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:49 2022] POST /identity/v3/projects => generated 377 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:49.962360 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:49.978430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ef89-272e-451c-9f0a-4cbecbaffa73 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:50.114382 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:50.115500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:50.116029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:50.116321 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:50.117166 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:50.119092 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:50.183306 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1368/2769] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:49 2022] POST /identity/v3/users => generated 295 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:50.187817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:50.203464 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52e6db0-d15e-484a-9d64-def86a1638e5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:50.209885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:50.211022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:50.211710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:50.212208 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:50.213679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:50.217073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:50.238595 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1402/2770] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:50.243326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:50.256099 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807adb6-5e6b-4ca0-aeca-aae914693243 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:50.263139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:50.264135 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:50.264421 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:50.265323 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] PATH_INFO: `/v3/projects/e456561d90e9484cab9c3851961821cc/users/743fbe23d591471085d43d2aff010d18/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:50.291648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e456561d90e9484cab9c3851961821cc, user_id=743fbe23d591471085d43d2aff010d18, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:50.294692 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:50.312927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1369/2771] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:50 2022] PUT /identity/v3/projects/e456561d90e9484cab9c3851961821cc/users/743fbe23d591471085d43d2aff010d18/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:50.316839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:50.337588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72581208-cf63-4c42-931f-e94d9a16d18b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:50.414952 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:50.416229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:50.416701 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:50.417215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:50.418271 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:50.420082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:50.432565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1403/2772] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:40:50.437357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:50.448563 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c04963-aa68-466d-b4db-31e438d4aad2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:50.453900 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bcc70-0c41-41ed-908f-30930f7ac365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:50.454675 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bcc70-0c41-41ed-908f-30930f7ac365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:50.454963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bcc70-0c41-41ed-908f-30930f7ac365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:50.455362 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bcc70-0c41-41ed-908f-30930f7ac365 admin admin] PATH_INFO: `/v3/projects/e456561d90e9484cab9c3851961821cc/users/743fbe23d591471085d43d2aff010d18/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:50.457808 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bcc70-0c41-41ed-908f-30930f7ac365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e456561d90e9484cab9c3851961821cc, user_id=743fbe23d591471085d43d2aff010d18, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:50.460413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443bcc70-0c41-41ed-908f-30930f7ac365 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:50.476484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1370/2773] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:40:50 2022] PUT /identity/v3/projects/e456561d90e9484cab9c3851961821cc/users/743fbe23d591471085d43d2aff010d18/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:52.743170 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:52.755416 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063280f5-785a-487d-b424-f550163951d2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:52.927505 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:52.928615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:52.928854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:52.929110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] PATH_INFO: `/v3/users/557f432d75114d028b0c81c43a44e38f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:52.944309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=557f432d75114d028b0c81c43a44e38f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:52.946466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:53.060108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:53.080374 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '557f432d75114d028b0c81c43a44e38f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:53.114495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '557f432d75114d028b0c81c43a44e38f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:53.115309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '557f432d75114d028b0c81c43a44e38f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:53.116092 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '557f432d75114d028b0c81c43a44e38f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:53.163697 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '557f432d75114d028b0c81c43a44e38f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:53.196211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1404/2774] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:52 2022] DELETE /identity/v3/users/557f432d75114d028b0c81c43a44e38f => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:53.201457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:53.202024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:53.202607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:53.237453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:53.241089 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:53.360221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:53.398746 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1371/2775] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:53.438173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:53.444767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:53.605969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b335778-5f1f-4241-a8ca-11120e57dfb6 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:53.616112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:53.616797 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:53.617084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:53.617363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:53.650489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:53.652089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:53.672270 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1405/2776] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:54.942719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:54.945004 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:54.972418 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9095009c-62de-4b42-8b09-7bfa52ce1da9 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:54.995423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:54.996681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:54.997203 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:54.997701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] PATH_INFO: `/v3/projects/e16441a7b0ca47318470f9e77f388f08` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:55.005917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e16441a7b0ca47318470f9e77f388f08)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:55.010821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:55.252918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e16441a7b0ca47318470f9e77f388f08', 'request_id': 'req-69e2e528-a546-473d-a096-297edeb81820'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.271157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e16441a7b0ca47318470f9e77f388f08', 'request_id': 'req-69e2e528-a546-473d-a096-297edeb81820'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.295229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e16441a7b0ca47318470f9e77f388f08', 'request_id': 'req-69e2e528-a546-473d-a096-297edeb81820'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.319684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] The token cache is being invalidate because project e16441a7b0ca47318470f9e77f388f08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:55.320143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.324949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.327140 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1372/2777] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:54 2022] DELETE /identity/v3/projects/e16441a7b0ca47318470f9e77f388f08 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:55.331055 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:55.341959 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2cd34d-674c-4b05-9c3d-7479735f2f1e service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:55.593638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:55.594558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:55.594844 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:55.595181 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] PATH_INFO: `/v3/users/ee1ffd6fb7074629b207c795d22537a0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:55.596571 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee1ffd6fb7074629b207c795d22537a0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:55.598742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:55.757580 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee1ffd6fb7074629b207c795d22537a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.772326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee1ffd6fb7074629b207c795d22537a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.774310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee1ffd6fb7074629b207c795d22537a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.775393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee1ffd6fb7074629b207c795d22537a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.783190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:55.803142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee1ffd6fb7074629b207c795d22537a0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:55.833738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1406/2778] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:55 2022] DELETE /identity/v3/users/ee1ffd6fb7074629b207c795d22537a0 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:57.052758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:57.054337 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:57.095837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2e528-a546-473d-a096-297edeb81820 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:57.300040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:57.300943 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:57.306759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:57.307063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] PATH_INFO: `/v3/projects/e23af21a25f741fa8871a19cd3239b09` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:57.312793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e23af21a25f741fa8871a19cd3239b09)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:57.315042 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:57.392931 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e23af21a25f741fa8871a19cd3239b09', 'request_id': 'req-f3e7f5d8-9fe1-4621-81e7-979d629e146d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.400038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e23af21a25f741fa8871a19cd3239b09', 'request_id': 'req-f3e7f5d8-9fe1-4621-81e7-979d629e146d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.409691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e23af21a25f741fa8871a19cd3239b09', 'request_id': 'req-f3e7f5d8-9fe1-4621-81e7-979d629e146d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.457425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] The token cache is being invalidate because project e23af21a25f741fa8871a19cd3239b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:57.457858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.459046 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.461212 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1373/2779] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:57 2022] DELETE /identity/v3/projects/e23af21a25f741fa8871a19cd3239b09 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:57.465113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:57.470992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:57.544334 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5324fe-16be-40f0-8953-191a455fde0c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:57.745991 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:57.747423 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:57.747839 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:57.748217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] PATH_INFO: `/v3/users/73c076e22520438c97e4660a629d0c58` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:57.753136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c076e22520438c97e4660a629d0c58)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:57.762691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:57.885140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c076e22520438c97e4660a629d0c58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.906661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c076e22520438c97e4660a629d0c58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.907501 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c076e22520438c97e4660a629d0c58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.908137 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c076e22520438c97e4660a629d0c58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.960358 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c076e22520438c97e4660a629d0c58'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:57.993442 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1407/2780] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:57 2022] DELETE /identity/v3/users/73c076e22520438c97e4660a629d0c58 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:58.471701 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:58.474521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:58.486583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:58.667148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:58.671363 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:58.695351 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e7f5d8-9fe1-4621-81e7-979d629e146d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:58.853625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:58.854235 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:58.854524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:58.854807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:58.857367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:58.879468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:58.881824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:58.895706 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:58.896453 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1374/2781] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:58.906987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda2bfb1-7b6b-4808-90e1-17a9a974751c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:58.915884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:58.918315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:58.918315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:58.918967 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] PATH_INFO: `/v3/users/09befefeb2e84ae79e7437b30cb3d908` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:58.921010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=09befefeb2e84ae79e7437b30cb3d908)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:58.923817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:58.986969 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:58.990434 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.021021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7591e1-2ce0-4b23-a587-b5c0628b0782 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:59.039688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09befefeb2e84ae79e7437b30cb3d908'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.042088 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:59.042983 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.043282 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.052726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] PATH_INFO: `/v3/projects/5394a542602d47ba9c661e4851154b28` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.054662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5394a542602d47ba9c661e4851154b28)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:59.056214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:59.083366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09befefeb2e84ae79e7437b30cb3d908'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.084250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09befefeb2e84ae79e7437b30cb3d908'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.084886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09befefeb2e84ae79e7437b30cb3d908'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.115428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-e92de106-6bc2-40f3-bbf0-0954eb8acc3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09befefeb2e84ae79e7437b30cb3d908'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.137454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1408/2782] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:40:58 2022] DELETE /identity/v3/users/09befefeb2e84ae79e7437b30cb3d908 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:59.165237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.165531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.165818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.209095 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] MFA Rules not processed for user `743fbe23d591471085d43d2aff010d18`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:59.210352 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.218824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5394a542602d47ba9c661e4851154b28', 'request_id': 'req-c2aa60a9-cba2-400b-81dd-edfeae0ab752'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.231545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5394a542602d47ba9c661e4851154b28', 'request_id': 'req-c2aa60a9-cba2-400b-81dd-edfeae0ab752'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.244385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5394a542602d47ba9c661e4851154b28', 'request_id': 'req-c2aa60a9-cba2-400b-81dd-edfeae0ab752'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.269861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] The token cache is being invalidate because project 5394a542602d47ba9c661e4851154b28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:59.270158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.271435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2aa60a9-cba2-400b-81dd-edfeae0ab752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.277125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1375/2783] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:58 2022] DELETE /identity/v3/projects/5394a542602d47ba9c661e4851154b28 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:40:59.287885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4f9d5-8eb0-4505-a79f-88c80e4233f7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.287885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4f9d5-8eb0-4505-a79f-88c80e4233f7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.288113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd4f9d5-8eb0-4505-a79f-88c80e4233f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.311544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-7cd4f9d5-8eb0-4505-a79f-88c80e4233f7 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:59.312998 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-7cd4f9d5-8eb0-4505-a79f-88c80e4233f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.351303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:59.408752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1409/2784] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:59.427943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:59.442688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.444641 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1376/2785] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:59.455067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.455540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.455939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.485057 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:59.486229 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.496648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1377/2786] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:59.502320 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:59.528747 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb55e1-554b-4f2e-9e73-6b720a8556d7 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:59.535221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:59.535830 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.536119 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.536400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.537136 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:40:59.537442 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:59.539625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:59.553040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1378/2787] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:40:59.558444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:59.560381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.568359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:40:59.574601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6b76a-ec12-4d60-9c60-6b9ccf0ddab3 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:59.584103 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:59.585058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.585342 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.585622 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] PATH_INFO: `/v3/projects/a1d7a585b8244bc5b2b2d8642b9160d0` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.587672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1d7a585b8244bc5b2b2d8642b9160d0)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:59.589628 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:59.590336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b59df0-2902-4796-8637-8959a1d4d400 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:59.609492 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:59.610215 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.612666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.612961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.616232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.662599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1d7a585b8244bc5b2b2d8642b9160d0', 'request_id': 'req-f56b2aa7-7215-43b6-b32d-3107ca4188c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.671959 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1d7a585b8244bc5b2b2d8642b9160d0', 'request_id': 'req-f56b2aa7-7215-43b6-b32d-3107ca4188c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.681329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1d7a585b8244bc5b2b2d8642b9160d0', 'request_id': 'req-f56b2aa7-7215-43b6-b32d-3107ca4188c0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.699976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:59.702346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ccf1a5-0b1b-4336-9ac5-f80a9b751e39 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:59.705019 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] The token cache is being invalidate because project a1d7a585b8244bc5b2b2d8642b9160d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:40:59.706030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.707457 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:40:59.709510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1379/2788] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:40:59 2022] DELETE /identity/v3/projects/a1d7a585b8244bc5b2b2d8642b9160d0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:40:59.710439 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:59.712167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.720267 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1410/2789] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:40:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:40:59.721752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87320eb0-dd9f-49c6-8fe6-164e5192a46d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.722110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87320eb0-dd9f-49c6-8fe6-164e5192a46d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.722403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87320eb0-dd9f-49c6-8fe6-164e5192a46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.744695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-87320eb0-dd9f-49c6-8fe6-164e5192a46d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:59.746124 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-87320eb0-dd9f-49c6-8fe6-164e5192a46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.808030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56b2aa7-7215-43b6-b32d-3107ca4188c0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:40:59.817896 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:40:59.818606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.818901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.819186 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.819977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:40:59.834040 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:40:59.885100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1411/2790] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:40:59.887458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:40:59.887458 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:40:59.887680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:40:59.893672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1380/2791] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/projects => generated 383 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:40:59.901287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:40:59.935125 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:40:59.936979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:40:59.952659 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:00.042889 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1412/2792] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:00.046851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.057501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aefff0-a5c3-400b-984e-0b575eca55c5 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.064601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.065284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.065577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.065858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.066702 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.067050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9e136-b5ae-4cc7-9be4-16e5ce108833 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.069567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.075928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.076626 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.077602 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.078005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.078637 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:41:00.083026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.086211 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.100443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1381/2793] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:40:59 2022] POST /identity/v3/users => generated 292 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:00.104837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.111311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1413/2794] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:41:00.112834 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.127101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe56d3-916d-4dbe-9ed9-986a923551f0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.127268 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec7b74-24be-4227-948c-b6d27441d7d8 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.136732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.138101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.149594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.149972 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.150839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.150987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.151681 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.151970 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.152260 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.153009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.153370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.155070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.167719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1382/2795] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:00.172343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.187322 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2484f1a-bb42-4681-a194-cc112785b9cc admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.193696 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.193999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.194567 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.194877 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.195802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.197930 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.210768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1383/2796] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:00.211468 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1414/2797] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:00 2022] POST /identity/v3/projects => generated 369 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:00.215527 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.225740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.228073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7139eb-4f73-4f8b-ad49-1a9b1af512a1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.246710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:00.325081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda559bf-5b7a-417d-97bc-a887796082d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.326973 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda559bf-5b7a-417d-97bc-a887796082d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.327406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda559bf-5b7a-417d-97bc-a887796082d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.327901 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda559bf-5b7a-417d-97bc-a887796082d3 admin admin] PATH_INFO: `/v3/projects/2b5c45d82d7643afac5d1c9ca622d94a/users/386f6e2319c848c0b831aaf503a7a911/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.354050 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda559bf-5b7a-417d-97bc-a887796082d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b5c45d82d7643afac5d1c9ca622d94a, user_id=386f6e2319c848c0b831aaf503a7a911, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.357348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda559bf-5b7a-417d-97bc-a887796082d3 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.376007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f8a36-0425-46f5-a236-04e87ffa7c14 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.381484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1384/2798] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:00 2022] PUT /identity/v3/projects/2b5c45d82d7643afac5d1c9ca622d94a/users/386f6e2319c848c0b831aaf503a7a911/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:00.393965 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.394476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.394861 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.437887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] MFA Rules not processed for user `386f6e2319c848c0b831aaf503a7a911`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:00.440459 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:00.459629 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.460461 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.460856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.461385 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.462291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.464361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.505273 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1415/2799] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:00 2022] POST /identity/v3/users => generated 285 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:00.510527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.528449 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ff3c0-7652-4e1d-a85d-7d918f0445ee admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.563954 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:00.570440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.571682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.573052 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.573342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.574136 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.578742 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.615815 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1385/2801] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:00.616021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1416/2801] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:00.616936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.619832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.640924 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078b7d7f-3505-453d-9077-9ac397907547 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.645559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dfd527-fcc2-433c-8f78-445f8fd67f05 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.654001 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.656588 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.656896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.657163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.657905 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.659397 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.660118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.660403 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.660681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.661388 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.663325 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.664992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.687321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1417/2802] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:00 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:00.691987 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.710854 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c774c-a0c8-48ab-b193-8e37aafe3411 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.727430 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1386/2803] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:00 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:00.728594 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:00.729708 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fe16c1-eef3-45f2-afe9-a15231f754b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:00.731108 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe16c1-eef3-45f2-afe9-a15231f754b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.732437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe16c1-eef3-45f2-afe9-a15231f754b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.734288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fe16c1-eef3-45f2-afe9-a15231f754b1 admin admin] PATH_INFO: `/v3/projects/f28180d0e3e64fc486e51ce9ecd35671/users/3d4bc16f62804a2e90ace639672cfbf8/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.771789 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe16c1-eef3-45f2-afe9-a15231f754b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f28180d0e3e64fc486e51ce9ecd35671, user_id=3d4bc16f62804a2e90ace639672cfbf8, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:00.776383 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fe16c1-eef3-45f2-afe9-a15231f754b1 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:00.805497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1418/2804] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:00 2022] PUT /identity/v3/projects/f28180d0e3e64fc486e51ce9ecd35671/users/3d4bc16f62804a2e90ace639672cfbf8/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:00.820045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:00.821448 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:00.822507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:00.891754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] MFA Rules not processed for user `3d4bc16f62804a2e90ace639672cfbf8`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:00.895808 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:00.900357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18fee40-59e9-4d56-a00a-7d07ab57f035 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:00.991851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:01.040498 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.041188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.041470 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.041748 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.042444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.044398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.050666 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1419/2805] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:00 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:01.066859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.087738 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c56f0be-c574-4b71-a987-4fbe9459e60b None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.093078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1387/2806] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:00 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:01.093078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.098613 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.099884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.100403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.100884 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.101796 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.104487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.114749 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729eda9a-dfd8-4c11-928c-c66388aa3022 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.127734 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.128484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.129033 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.129333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.130031 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.133655 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.142284 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1420/2807] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:01 2022] POST /identity/v3/projects => generated 369 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:01.146647 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.165859 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1388/2808] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:01.171220 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.194693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89e386-4144-475a-8e5f-b303e516f7d9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.245912 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e91b139-5c7e-4b9e-8624-d8e4fbdd5816 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.254635 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.256364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.256707 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.256994 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.257695 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.261757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.323933 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1421/2809] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:01 2022] POST /identity/v3/users => generated 291 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:01.325194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:01.328222 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.344283 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.345126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.345308 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.345541 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.346303 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.347967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.367431 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8e4766-26dd-4b41-a788-4187481bbe56 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.379317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1389/2810] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:01.382677 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.391172 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.391927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.392223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.392494 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.393229 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.395210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.408793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba00ae-87f6-4c0f-b5fb-8a2c737ee9ca admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.411454 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1422/2811] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:01.412409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.423435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef8fcc-8512-40b3-9143-f76d89c0f109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.424368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef8fcc-8512-40b3-9143-f76d89c0f109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.424654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef8fcc-8512-40b3-9143-f76d89c0f109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.424935 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef8fcc-8512-40b3-9143-f76d89c0f109 admin admin] PATH_INFO: `/v3/projects/c32c753bd3ea4f59965108ec2c6276a1/users/39a79d0f4f4d42ef862c598e02c99a21/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.431127 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea852f8d-fb32-4a16-acd1-505873b30a59 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.449585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.450242 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.450531 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.450813 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] PATH_INFO: `/v3/projects/2e37a73eb1d44b049c1b4580b5c783b3/users/baac69617b26488596a4098e01848acf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.466706 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef8fcc-8512-40b3-9143-f76d89c0f109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c32c753bd3ea4f59965108ec2c6276a1, user_id=39a79d0f4f4d42ef862c598e02c99a21, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.471797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bef8fcc-8512-40b3-9143-f76d89c0f109 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.483679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e37a73eb1d44b049c1b4580b5c783b3, user_id=baac69617b26488596a4098e01848acf, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.487584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.504672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1390/2812] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:01 2022] PUT /identity/v3/projects/c32c753bd3ea4f59965108ec2c6276a1/users/39a79d0f4f4d42ef862c598e02c99a21/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:01.511718 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1423/2813] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:01 2022] PUT /identity/v3/projects/2e37a73eb1d44b049c1b4580b5c783b3/users/baac69617b26488596a4098e01848acf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:01.515875 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.516347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.516982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.521860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.589516 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] MFA Rules not processed for user `39a79d0f4f4d42ef862c598e02c99a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:01.591105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:01.717420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8f4dda-05ad-4e11-8cb8-5a12a18b7954 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.729078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.730942 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.731261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.733405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.734301 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.735980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.767316 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1424/2814] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:01.772359 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.773017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1391/2815] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:01.774367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.790824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4d029-c8f1-4e5a-b1f9-0a28bcde5c4a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.795073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244212ce-443e-49b0-ad2e-f93bbbb9096d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:01.802026 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0676690b-b274-4fae-80c6-f123dca72ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.802945 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676690b-b274-4fae-80c6-f123dca72ff5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.804370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676690b-b274-4fae-80c6-f123dca72ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.804680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0676690b-b274-4fae-80c6-f123dca72ff5 admin admin] PATH_INFO: `/v3/projects/2e37a73eb1d44b049c1b4580b5c783b3/users/baac69617b26488596a4098e01848acf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.806163 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:01.807449 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.807740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.808026 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.808729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.810648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.813182 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676690b-b274-4fae-80c6-f123dca72ff5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e37a73eb1d44b049c1b4580b5c783b3, user_id=baac69617b26488596a4098e01848acf, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:01.819705 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0676690b-b274-4fae-80c6-f123dca72ff5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:01.838302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:01.854682 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1425/2816] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:01 2022] PUT /identity/v3/projects/2e37a73eb1d44b049c1b4580b5c783b3/users/baac69617b26488596a4098e01848acf/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:01.867173 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:01.867554 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:01.868926 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:01.923807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1392/2817] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:01 2022] POST /identity/v3/projects => generated 383 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:01.925999 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:01.931007 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] MFA Rules not processed for user `baac69617b26488596a4098e01848acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:01.932326 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:02.098821 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592d43-4da1-4f87-b840-911a4a30782a admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.099380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1426/2818] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:01 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:02.102720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.103595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.103595 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.103872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.104593 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.106537 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.113089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:02.114532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:02.158063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1393/2819] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:01 2022] POST /identity/v3/users => generated 298 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:02.167504 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:02.189577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802bee59-d97e-4a0e-a319-73cb776bb8d2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.216433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.218268 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.218744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.219304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.222500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.229222 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.260679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b694589c-747f-46aa-8cb6-809671657fbc None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.266006 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.267811 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1394/2820] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:02.269785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.270082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.270364 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.270469 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:02.298548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.300233 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.300465 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c261fc-b97d-4401-90d0-60da1b7d0dc2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.307155 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.308137 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.309296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.309645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] PATH_INFO: `/v3/projects/6d0568b5c12a45f8ab5509ff7de64d9b/users/83861e7ea2824659bfb1c73e90bff275/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.321972 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1427/2821] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:41:02.350032 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:02.383980 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0568b5c12a45f8ab5509ff7de64d9b, user_id=83861e7ea2824659bfb1c73e90bff275, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.392432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.395010 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:02.406628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741e6750-4310-4196-b090-3df7725af337 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.414740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.415412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.415704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.415986 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.455621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1395/2822] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:02 2022] PUT /identity/v3/projects/6d0568b5c12a45f8ab5509ff7de64d9b/users/83861e7ea2824659bfb1c73e90bff275/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:02.462178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:02.585740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.587663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.613035 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1428/2823] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:02 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:41:02.652848 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df563ba4-0843-4fbc-84b3-3043eef90fbe admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.663824 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b43c970-71a8-4b5e-998e-a1475873dc07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.664988 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43c970-71a8-4b5e-998e-a1475873dc07 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.665285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43c970-71a8-4b5e-998e-a1475873dc07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.665557 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b43c970-71a8-4b5e-998e-a1475873dc07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.667338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43c970-71a8-4b5e-998e-a1475873dc07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.671759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b43c970-71a8-4b5e-998e-a1475873dc07 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.702307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1396/2824] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:02 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:02.707427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:02.722709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17607c60-ceee-4210-b02e-3955664ce605 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:02.729692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:02.731066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.731527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.732078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] PATH_INFO: `/v3/projects/6d0568b5c12a45f8ab5509ff7de64d9b/users/83861e7ea2824659bfb1c73e90bff275/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.735151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0568b5c12a45f8ab5509ff7de64d9b, user_id=83861e7ea2824659bfb1c73e90bff275, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:02.739276 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:02.753380 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1429/2825] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:02 2022] PUT /identity/v3/projects/6d0568b5c12a45f8ab5509ff7de64d9b/users/83861e7ea2824659bfb1c73e90bff275/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:41:02.764708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:02.765782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:02.766258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:02.813129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] MFA Rules not processed for user `83861e7ea2824659bfb1c73e90bff275`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:02.841355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:03.013897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1397/2826] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:03.031512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:03.033234 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:03.053005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:03.171714 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daede720-0bc0-4b74-b74f-2a36c70fcce5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:03.177667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:03.178275 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:03.178638 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:03.178904 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:03.258082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:03.259857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:03.289031 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1430/2827] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:04.939078 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:04.949079 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a3e27-c62e-46f3-a6bb-c728dbdf6d3e None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:04.953151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:04.953784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:04.954066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:04.954340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:05.015917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:05.017346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:05.026447 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1398/2828] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:05.595636 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:05.604895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f959ca-7542-4802-ad8f-ec1a16d8c5d3 service cinder] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:05.614825 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa41ae1-76f6-4725-8a28-a36e2142f90c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:05.615627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41ae1-76f6-4725-8a28-a36e2142f90c service cinder] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:05.615982 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41ae1-76f6-4725-8a28-a36e2142f90c service cinder] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:05.616331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa41ae1-76f6-4725-8a28-a36e2142f90c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:05.626820 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41ae1-76f6-4725-8a28-a36e2142f90c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:05.628421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa41ae1-76f6-4725-8a28-a36e2142f90c service cinder] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:05.638346 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1431/2829] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:06.283835 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:06.293051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:06.353538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf81814d-16ff-4973-872a-07b8c2421d98 service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:06.360505 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:06.361404 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:06.361691 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:06.361967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] PATH_INFO: `/v3/users/3d4bc16f62804a2e90ace639672cfbf8` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:06.363207 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4bc16f62804a2e90ace639672cfbf8)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:06.365309 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:06.415490 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4bc16f62804a2e90ace639672cfbf8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:06.422927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4bc16f62804a2e90ace639672cfbf8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:06.423695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4bc16f62804a2e90ace639672cfbf8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:06.424317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4bc16f62804a2e90ace639672cfbf8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:06.436654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4bc16f62804a2e90ace639672cfbf8'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:06.451129 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1399/2830] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:06 2022] DELETE /identity/v3/users/3d4bc16f62804a2e90ace639672cfbf8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:06.455781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:06.456079 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:06.456361 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:06.473280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:06.474751 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:06.536523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1432/2831] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:06.546817 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:06.548407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:06.614174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2298c43-5dd2-4647-a43d-c78ca27c75cb admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:06.617857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:06.618506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:06.618794 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:06.619076 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:06.628357 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:06.629963 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:06.641729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1400/2832] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:06.966481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:06.969754 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:07.002968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a7c27-7fee-4f9c-9928-aede6f6291f1 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:07.008244 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:07.009250 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:07.009534 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:07.009822 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] PATH_INFO: `/v3/projects/f28180d0e3e64fc486e51ce9ecd35671` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:07.011102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f28180d0e3e64fc486e51ce9ecd35671)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:07.013015 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:07.081372 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f28180d0e3e64fc486e51ce9ecd35671', 'request_id': 'req-07535077-48e2-4bed-8318-bf1bd0d3cac4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.082877 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:07.089333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f28180d0e3e64fc486e51ce9ecd35671', 'request_id': 'req-07535077-48e2-4bed-8318-bf1bd0d3cac4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.097651 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f28180d0e3e64fc486e51ce9ecd35671', 'request_id': 'req-07535077-48e2-4bed-8318-bf1bd0d3cac4'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.115680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] The token cache is being invalidate because project f28180d0e3e64fc486e51ce9ecd35671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:07.116369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.117141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.118903 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1433/2833] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:06 2022] DELETE /identity/v3/projects/f28180d0e3e64fc486e51ce9ecd35671 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:07.122542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:07.136243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016a6135-4f03-41a8-a69c-a5620a4399b5 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:07.173714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:07.232440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:07.233368 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:07.233654 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:07.237801 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] PATH_INFO: `/v3/users/baac69617b26488596a4098e01848acf` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:07.238661 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baac69617b26488596a4098e01848acf)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:07.240914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:07.295546 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baac69617b26488596a4098e01848acf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.302572 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baac69617b26488596a4098e01848acf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.303460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baac69617b26488596a4098e01848acf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.304075 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baac69617b26488596a4098e01848acf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.316500 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baac69617b26488596a4098e01848acf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.329865 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1401/2834] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:07 2022] DELETE /identity/v3/users/baac69617b26488596a4098e01848acf => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:07.625427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:07.628114 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:07.660404 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07535077-48e2-4bed-8318-bf1bd0d3cac4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:07.667160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:07.668620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:07.745090 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:07.746116 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:07.746427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:07.746816 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] PATH_INFO: `/v3/projects/2e37a73eb1d44b049c1b4580b5c783b3` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:07.749249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e37a73eb1d44b049c1b4580b5c783b3)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:07.751201 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:07.764625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7997d9-7ae4-4d27-ba83-81f9106297d9 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:07.770167 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:07.770897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:07.771304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:07.771660 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:07.773510 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:07.786695 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:07.788448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06610595-e8d6-4492-9d4b-22ea68424e8f service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:07.808885 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1402/2835] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:07.809424 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e37a73eb1d44b049c1b4580b5c783b3', 'request_id': 'req-cdec749a-7009-4abf-98c8-36b9b7dbb221'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.816420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e37a73eb1d44b049c1b4580b5c783b3', 'request_id': 'req-cdec749a-7009-4abf-98c8-36b9b7dbb221'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.822862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e37a73eb1d44b049c1b4580b5c783b3', 'request_id': 'req-cdec749a-7009-4abf-98c8-36b9b7dbb221'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.837237 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] The token cache is being invalidate because project 2e37a73eb1d44b049c1b4580b5c783b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:07.837529 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.838269 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-cdec749a-7009-4abf-98c8-36b9b7dbb221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:07.840180 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1434/2836] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:07 2022] DELETE /identity/v3/projects/2e37a73eb1d44b049c1b4580b5c783b3 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:07.845667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:07.845955 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:07.846238 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:07.865906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:07.867232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:07.903781 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:07.961869 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1403/2837] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:07.972860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:07.973332 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:07.973480 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.001189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:08.002818 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:08.010563 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1435/2838] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:08.014989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.025399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a7c2d-af11-4c67-b6ce-d1e71f77c5ed None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.030797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.031486 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.031795 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.032072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.032804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80161) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:41:08.033117 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.035292 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.051070 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1404/2839] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:41:08.055289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.064217 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3885f4b2-be5c-40a1-a9a3-3b210d2aef1d None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.069040 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.069801 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.070153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.070499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.071315 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.073321 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.104030 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1436/2840] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:08 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:08.105221 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.116017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d3ba8-05aa-4182-9ce3-bc6a883b5f19 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.178433 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.179116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.179506 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.179906 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.180711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.182792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.204719 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1405/2841] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:08 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:08.208729 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.220021 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52888fa7-75b8-4c3b-bb56-db06bb2d51fc admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.225050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.225722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.226011 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.226291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.227020 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.228668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.240542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1437/2842] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:08.244664 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.260853 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6426f5-866b-4a81-b9e9-5bdfe1d6d984 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.262312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc39aa5-d63c-4565-82b4-abf00ecaf496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.263194 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc39aa5-d63c-4565-82b4-abf00ecaf496 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.263509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc39aa5-d63c-4565-82b4-abf00ecaf496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.264009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc39aa5-d63c-4565-82b4-abf00ecaf496 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.264964 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc39aa5-d63c-4565-82b4-abf00ecaf496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.266711 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc39aa5-d63c-4565-82b4-abf00ecaf496 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.282125 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1406/2843] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:41:08.286420 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.299150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb1d26-c128-4943-9553-54132ca9c0c0 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.304278 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.305302 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.305687 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.306009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] PATH_INFO: `/v3/projects/5165892050ee4c099b9a08fc8a9ce7f5/users/c659618d17e546749f59f8e576602962/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.333598 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5165892050ee4c099b9a08fc8a9ce7f5, user_id=c659618d17e546749f59f8e576602962, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.336432 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.352559 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1438/2844] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:08 2022] PUT /identity/v3/projects/5165892050ee4c099b9a08fc8a9ce7f5/users/c659618d17e546749f59f8e576602962/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:08.364387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.364819 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.368343 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.413276 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] MFA Rules not processed for user `c659618d17e546749f59f8e576602962`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:08.414492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:08.508681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1407/2845] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:08.513342 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.522860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:08.608200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01baf1d5-083f-4905-8d40-01a60527f03e admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.613109 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.613830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.614113 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.614439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.615151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.617230 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.653121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1439/2846] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:08 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:08.663472 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.674508 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2777668-26a2-416b-aaaa-77c0ad8083a3 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.707190 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:08.736693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.737378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.737671 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.737979 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.738657 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.740606 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.761758 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1408/2847] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:08 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:08.766585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.777440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d580-fc09-432c-9a65-846a6fcafc2b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.782817 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.784483 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.784923 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.785403 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.786146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.788070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.800775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1440/2848] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:08.805242 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.818139 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729b921-846d-436c-badf-3b92bf3d52f1 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.831863 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df32bd58-c815-4421-a7cd-2a6edfeb9e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.832538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32bd58-c815-4421-a7cd-2a6edfeb9e25 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.833161 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32bd58-c815-4421-a7cd-2a6edfeb9e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.833568 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32bd58-c815-4421-a7cd-2a6edfeb9e25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.834366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32bd58-c815-4421-a7cd-2a6edfeb9e25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.836239 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df32bd58-c815-4421-a7cd-2a6edfeb9e25 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.847540 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1409/2849] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:08.852597 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:08.864863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53cde3-575a-4aaf-a68f-871250b53ddb admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:08.870668 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:08.871806 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.872018 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.872307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] PATH_INFO: `/v3/projects/340731dee6fd4572936af21e84e81467/users/aca6e21ace6546d49b413c972768e45f/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.893112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=340731dee6fd4572936af21e84e81467, user_id=aca6e21ace6546d49b413c972768e45f, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:08.896089 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:08.916745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1441/2850] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:08 2022] PUT /identity/v3/projects/340731dee6fd4572936af21e84e81467/users/aca6e21ace6546d49b413c972768e45f/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:41:08.927784 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:08.928174 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:08.928501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:08.960729 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] MFA Rules not processed for user `aca6e21ace6546d49b413c972768e45f`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:08.961960 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:09.045215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1410/2851] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:09.049573 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.121333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7973-3da7-4dd1-94d3-5b90282927a5 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.126711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.130046 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.130336 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.130621 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] PATH_INFO: `/v3/projects` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.131393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.133356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.161174 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1442/2852] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:09 2022] POST /identity/v3/projects => generated 379 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:09.165366 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.176902 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90634d3-9f2c-4888-ad23-481dc828f56b None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.237621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.238304 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.238592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.238879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] PATH_INFO: `/v3/users` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.239625 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.241591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.270432 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1411/2853] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:09 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:09.275221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.286047 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2f2c71-bc12-4ac3-bcb8-5882c643259b admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.290856 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.291581 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.291874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.292158 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.292887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.294538 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.307081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1443/2854] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:09.312296 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.325210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b06c2-6478-4974-8f0b-40f46805ec7d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.329967 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.330871 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.331160 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.331550 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] PATH_INFO: `/v3/projects/6c027fd5054d44b4a52d36a6a27bb9ff/users/e6e0f2f92a174c959a74c844372f13fc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.348529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c027fd5054d44b4a52d36a6a27bb9ff, user_id=e6e0f2f92a174c959a74c844372f13fc, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.351499 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.366208 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1412/2855] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:09 2022] PUT /identity/v3/projects/6c027fd5054d44b4a52d36a6a27bb9ff/users/e6e0f2f92a174c959a74c844372f13fc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:09.370386 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.382547 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a467893-d4b9-49e9-bd71-786f34e9c130 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.410740 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:09.444451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797413ae-4f3c-4178-816a-2635cea96d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.445121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797413ae-4f3c-4178-816a-2635cea96d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.445437 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797413ae-4f3c-4178-816a-2635cea96d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.445720 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797413ae-4f3c-4178-816a-2635cea96d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.446440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797413ae-4f3c-4178-816a-2635cea96d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.448256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797413ae-4f3c-4178-816a-2635cea96d73 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.458727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1444/2856] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:09.463455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.474607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b208eb-8a3c-4655-af28-e580677f6510 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.479400 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.480359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.480720 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.481073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] PATH_INFO: `/v3/projects/6c027fd5054d44b4a52d36a6a27bb9ff/users/e6e0f2f92a174c959a74c844372f13fc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.483684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c027fd5054d44b4a52d36a6a27bb9ff, user_id=e6e0f2f92a174c959a74c844372f13fc, role_id=f7649bd8aa81487b8ca1fbc0e5fcba91)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.486370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.488708 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:09.498977 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1413/2857] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:09 2022] PUT /identity/v3/projects/6c027fd5054d44b4a52d36a6a27bb9ff/users/e6e0f2f92a174c959a74c844372f13fc/roles/f7649bd8aa81487b8ca1fbc0e5fcba91 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:09.563512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.563829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.564112 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.594439 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] MFA Rules not processed for user `e6e0f2f92a174c959a74c844372f13fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:09.595865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:09.674484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1445/2858] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:09.691173 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.693087 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:09.743146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.744727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:09.766232 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704e33c-c414-414f-a6c6-e621f272489d admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.771561 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.772658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.773069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.773355 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.785681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.789926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.807575 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1414/2859] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:09.833799 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6e02c-5913-47ad-b800-eebc70f6c58c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.839793 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.840896 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.841317 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.841747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] PATH_INFO: `/v3/users/386f6e2319c848c0b831aaf503a7a911` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.843760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=386f6e2319c848c0b831aaf503a7a911)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.845727 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.851941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:09.855834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:09.869206 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c33586c-4683-415b-bd6c-c26b44dc9d4e service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:09.873317 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff2edc4-182a-420d-881a-3f65361829cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:09.874041 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2edc4-182a-420d-881a-3f65361829cb service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.874525 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2edc4-182a-420d-881a-3f65361829cb service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.874918 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff2edc4-182a-420d-881a-3f65361829cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.911615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '386f6e2319c848c0b831aaf503a7a911'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:09.919199 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:09.922886 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '386f6e2319c848c0b831aaf503a7a911'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:09.923931 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '386f6e2319c848c0b831aaf503a7a911'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:09.925028 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '386f6e2319c848c0b831aaf503a7a911'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:09.941784 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '386f6e2319c848c0b831aaf503a7a911'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:09.942214 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2edc4-182a-420d-881a-3f65361829cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:09.943760 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff2edc4-182a-420d-881a-3f65361829cb service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:09.953822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1415/2860] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:09.961688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1446/2861] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:09 2022] DELETE /identity/v3/users/386f6e2319c848c0b831aaf503a7a911 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:09.970071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:09.971929 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:09.972413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:09.993494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:09.994820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:10.084927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1416/2862] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:10.086485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:10.087992 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:10.159527 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505eae6-54aa-4ba6-826c-637c75e8be3d admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:10.167830 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:10.168536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:10.168824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:10.169104 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:10.179824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:10.181256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:10.193311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1447/2863] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:10.781923 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:10.783721 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:10.796807 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db5f15-590e-450d-b140-0547b4a05d8a None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:10.804027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:10.805118 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:10.805481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:10.807224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] PATH_INFO: `/v3/projects/2b5c45d82d7643afac5d1c9ca622d94a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:10.808674 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b5c45d82d7643afac5d1c9ca622d94a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:10.812620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:10.860957 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:10.875893 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b5c45d82d7643afac5d1c9ca622d94a', 'request_id': 'req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:10.888761 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b5c45d82d7643afac5d1c9ca622d94a', 'request_id': 'req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:10.896157 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b5c45d82d7643afac5d1c9ca622d94a', 'request_id': 'req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:10.909529 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] The token cache is being invalidate because project 2b5c45d82d7643afac5d1c9ca622d94a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:10.909818 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:10.910461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:10.912284 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1417/2864] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:10 2022] DELETE /identity/v3/projects/2b5c45d82d7643afac5d1c9ca622d94a => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:41:10.916266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:10.926960 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a800fcf1-f79a-4219-bd0b-a4d184b22135 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:10.992660 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:10.993579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:10.993869 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:10.994152 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] PATH_INFO: `/v3/users/39a79d0f4f4d42ef862c598e02c99a21` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:10.995409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39a79d0f4f4d42ef862c598e02c99a21)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:10.997592 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:11.055295 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39a79d0f4f4d42ef862c598e02c99a21'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.062369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39a79d0f4f4d42ef862c598e02c99a21'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.063401 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39a79d0f4f4d42ef862c598e02c99a21'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.064282 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39a79d0f4f4d42ef862c598e02c99a21'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.078940 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39a79d0f4f4d42ef862c598e02c99a21'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.080864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:11.095814 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1448/2865] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:10 2022] DELETE /identity/v3/users/39a79d0f4f4d42ef862c598e02c99a21 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:11.415126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:11.416604 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:11.434777 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db16d7b-bb1c-4e86-866e-fafc6dc9438f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:11.514856 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:11.515872 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:11.516312 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:11.516601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] PATH_INFO: `/v3/projects/c32c753bd3ea4f59965108ec2c6276a1` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:11.518016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c32c753bd3ea4f59965108ec2c6276a1)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:11.523686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:11.585970 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c32c753bd3ea4f59965108ec2c6276a1', 'request_id': 'req-dbe27c67-7035-43a7-9e1c-491dbe9f5482'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.595450 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c32c753bd3ea4f59965108ec2c6276a1', 'request_id': 'req-dbe27c67-7035-43a7-9e1c-491dbe9f5482'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.602532 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c32c753bd3ea4f59965108ec2c6276a1', 'request_id': 'req-dbe27c67-7035-43a7-9e1c-491dbe9f5482'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.630583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] The token cache is being invalidate because project c32c753bd3ea4f59965108ec2c6276a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:11.630583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.630583 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.631837 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1418/2866] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:11 2022] DELETE /identity/v3/projects/c32c753bd3ea4f59965108ec2c6276a1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:11.635617 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:11.637153 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:11.650495 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc1b4cd-6748-40d1-90e0-0ebe48dba8a3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:11.723791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:11.725008 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:11.725366 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:11.725785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] PATH_INFO: `/v3/users/83861e7ea2824659bfb1c73e90bff275` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:11.730526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83861e7ea2824659bfb1c73e90bff275)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:11.730526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:11.825016 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83861e7ea2824659bfb1c73e90bff275'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.834979 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83861e7ea2824659bfb1c73e90bff275'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.835764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83861e7ea2824659bfb1c73e90bff275'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.836466 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83861e7ea2824659bfb1c73e90bff275'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.850290 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-920fe152-0bf5-4755-a023-622a7d42df47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83861e7ea2824659bfb1c73e90bff275'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:11.867119 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1449/2867] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:11 2022] DELETE /identity/v3/users/83861e7ea2824659bfb1c73e90bff275 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:12.190549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.192656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:12.219016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe27c67-7035-43a7-9e1c-491dbe9f5482 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.277771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:12.295412 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.296300 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.296591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.296876 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] PATH_INFO: `/v3/projects/6d0568b5c12a45f8ab5509ff7de64d9b` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.298240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0568b5c12a45f8ab5509ff7de64d9b)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.300476 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.359109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0568b5c12a45f8ab5509ff7de64d9b', 'request_id': 'req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:12.365000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0568b5c12a45f8ab5509ff7de64d9b', 'request_id': 'req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:12.371380 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0568b5c12a45f8ab5509ff7de64d9b', 'request_id': 'req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:12.385544 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] The token cache is being invalidate because project 6d0568b5c12a45f8ab5509ff7de64d9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:12.385828 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:12.386451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-57d91b3a-ef4c-4408-98b1-9acde3a9f0eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:12.388029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1419/2868] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:12 2022] DELETE /identity/v3/projects/6d0568b5c12a45f8ab5509ff7de64d9b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:12.393653 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.394142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.394235 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.413512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:12.414747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:12.482248 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1450/2869] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:12.484154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.484626 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.485187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.504410 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:12.505645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:12.514694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1420/2870] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:12.519121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.530151 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51e65c-96d3-4876-a96a-af5eaefc9658 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.535076 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.535778 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.536061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.536333 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.537062 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:41:12.537369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.539009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.551067 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:12.554421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1451/2871] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:41:12.558855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.569347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e83815-c9f7-41a3-ac9a-60def106d438 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.574048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.574722 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.575012 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.575334 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.576048 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.578016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.610976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1421/2872] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:12 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:12.612400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.622628 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e6f04c-9afc-442c-9c0b-b813610b31a4 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.680369 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.681102 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.681392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.681675 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.682392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.684395 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.707913 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1452/2873] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:12 2022] POST /identity/v3/users => generated 284 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 20 06:41:12.709143 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.720328 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9c2dca-34f0-47bf-8a3e-490e5c0994c0 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.725340 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.726008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.726293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.726693 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.727460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.729069 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.744054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1422/2874] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:12.745859 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.756264 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab45c5da-e845-4e80-a943-decc91c7efcf admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.761005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5589b4e6-0ab4-4397-8f92-ab916a90098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.761704 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b4e6-0ab4-4397-8f92-ab916a90098d admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.761997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b4e6-0ab4-4397-8f92-ab916a90098d admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.762287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5589b4e6-0ab4-4397-8f92-ab916a90098d admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.763022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b4e6-0ab4-4397-8f92-ab916a90098d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.764709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5589b4e6-0ab4-4397-8f92-ab916a90098d admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.774927 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1453/2875] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:12.779116 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.793895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab5d5e-e973-40f4-a43c-32e637abe973 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:12.795911 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:12.796971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.797258 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.797530 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] PATH_INFO: `/v3/projects/74a4a77a30cb4b32b481bfd7a56bdba5/users/9afdb957fe8f4cf88bcdb14450abce7a/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.813993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74a4a77a30cb4b32b481bfd7a56bdba5, user_id=9afdb957fe8f4cf88bcdb14450abce7a, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:12.822686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:12.837233 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1423/2876] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:12 2022] PUT /identity/v3/projects/74a4a77a30cb4b32b481bfd7a56bdba5/users/9afdb957fe8f4cf88bcdb14450abce7a/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:12.848785 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:12.849419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:12.849866 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:12.880356 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] MFA Rules not processed for user `9afdb957fe8f4cf88bcdb14450abce7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:12.881663 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:12.946518 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1454/2877] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:12 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:12.958460 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:12.964768 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:13.032381 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd161d5e-048d-4309-a70d-2f8f6e14e851 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:13.036319 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d10b04a-b673-4991-8c12-b4d97fc4442c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '89c777401892485099d5448c506debb0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:13.037104 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10b04a-b673-4991-8c12-b4d97fc4442c service glance] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:13.037638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10b04a-b673-4991-8c12-b4d97fc4442c service glance] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:13.038018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d10b04a-b673-4991-8c12-b4d97fc4442c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:13.047542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:13.048700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10b04a-b673-4991-8c12-b4d97fc4442c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:13.048943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:13.050359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d10b04a-b673-4991-8c12-b4d97fc4442c service glance] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:13.060744 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1424/2878] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:12 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:13.126608 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf755f-d230-4027-a0b4-baceba5010d5 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:13.135077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:13.135899 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:13.136148 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:13.136430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] PATH_INFO: `/v3/users/c659618d17e546749f59f8e576602962` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:13.137853 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c659618d17e546749f59f8e576602962)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:13.140640 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:13.148195 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:13.205887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c659618d17e546749f59f8e576602962'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.213410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c659618d17e546749f59f8e576602962'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.214213 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c659618d17e546749f59f8e576602962'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.214921 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c659618d17e546749f59f8e576602962'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.230584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c659618d17e546749f59f8e576602962'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.259993 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1455/2879] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:13 2022] DELETE /identity/v3/users/c659618d17e546749f59f8e576602962 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:41:13.264926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:13.265924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:13.266415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:13.286438 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:13.288584 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:13.302359 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:13.378387 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1425/2880] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:13.391790 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:13.393831 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:13.459479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b272be-dddf-4c3a-9624-74bd190a334c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:13.464410 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:13.465435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:13.465879 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:13.466288 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:13.477470 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:13.479289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:13.490188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1456/2881] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:13.798714 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:13.800608 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:13.813658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b970cd-2006-4f4a-b72c-a1b209e6cd6d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:13.833091 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:13.834067 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:13.834356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:13.834638 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] PATH_INFO: `/v3/projects/5165892050ee4c099b9a08fc8a9ce7f5` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:13.836072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5165892050ee4c099b9a08fc8a9ce7f5)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:13.838073 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:13.890197 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5165892050ee4c099b9a08fc8a9ce7f5', 'request_id': 'req-bdd816c3-8cda-4e0b-8582-42dc0595332f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.896598 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5165892050ee4c099b9a08fc8a9ce7f5', 'request_id': 'req-bdd816c3-8cda-4e0b-8582-42dc0595332f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.904105 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5165892050ee4c099b9a08fc8a9ce7f5', 'request_id': 'req-bdd816c3-8cda-4e0b-8582-42dc0595332f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.917333 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] The token cache is being invalidate because project 5165892050ee4c099b9a08fc8a9ce7f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:13.917621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.918240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:13.920352 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1426/2882] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:13 2022] DELETE /identity/v3/projects/5165892050ee4c099b9a08fc8a9ce7f5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:13.924291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:13.934419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be9fbb-5553-4247-8dcf-45cf4fad38b7 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:13.998978 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:14.000254 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:14.000823 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:14.001256 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] PATH_INFO: `/v3/users/aca6e21ace6546d49b413c972768e45f` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:14.002497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aca6e21ace6546d49b413c972768e45f)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:14.004745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:14.040936 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:14.056862 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aca6e21ace6546d49b413c972768e45f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.064061 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aca6e21ace6546d49b413c972768e45f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.064760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aca6e21ace6546d49b413c972768e45f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.065367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aca6e21ace6546d49b413c972768e45f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.077864 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aca6e21ace6546d49b413c972768e45f'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.097457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1457/2883] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:13 2022] DELETE /identity/v3/users/aca6e21ace6546d49b413c972768e45f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:14.423141 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:14.424726 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:14.440686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd816c3-8cda-4e0b-8582-42dc0595332f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:14.506832 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:14.507852 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:14.508127 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:14.508431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] PATH_INFO: `/v3/projects/340731dee6fd4572936af21e84e81467` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:14.509617 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=340731dee6fd4572936af21e84e81467)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:14.511914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:14.557552 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:14.562526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '340731dee6fd4572936af21e84e81467', 'request_id': 'req-91e23859-bd6f-4e49-b271-34d2a45b3941'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.569086 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '340731dee6fd4572936af21e84e81467', 'request_id': 'req-91e23859-bd6f-4e49-b271-34d2a45b3941'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.575379 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '340731dee6fd4572936af21e84e81467', 'request_id': 'req-91e23859-bd6f-4e49-b271-34d2a45b3941'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.592651 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] The token cache is being invalidate because project 340731dee6fd4572936af21e84e81467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:14.592940 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.593570 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.595131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1427/2884] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:14 2022] DELETE /identity/v3/projects/340731dee6fd4572936af21e84e81467 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:14.599757 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:14.601311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:14.616425 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226640e9-bfb1-4649-bd50-71c08a8c2410 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:14.687484 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:14.688370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:14.688658 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:14.688938 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] PATH_INFO: `/v3/users/e6e0f2f92a174c959a74c844372f13fc` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:14.690150 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e0f2f92a174c959a74c844372f13fc)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:14.692140 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:14.742857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e0f2f92a174c959a74c844372f13fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.750106 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e0f2f92a174c959a74c844372f13fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.750779 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e0f2f92a174c959a74c844372f13fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.751421 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e0f2f92a174c959a74c844372f13fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.763353 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-c4fe692f-59eb-410b-94d1-321d67b3683a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e0f2f92a174c959a74c844372f13fc'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:14.790526 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1458/2885] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:14 2022] DELETE /identity/v3/users/e6e0f2f92a174c959a74c844372f13fc => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:15.086666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.088156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:15.116461 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e23859-bd6f-4e49-b271-34d2a45b3941 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.175243 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.176187 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.176473 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.176751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] PATH_INFO: `/v3/projects/6c027fd5054d44b4a52d36a6a27bb9ff` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.177939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c027fd5054d44b4a52d36a6a27bb9ff)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.179887 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.228126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c027fd5054d44b4a52d36a6a27bb9ff', 'request_id': 'req-6232ae39-fc4b-4840-a06c-d3e4a59795a0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:15.234217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c027fd5054d44b4a52d36a6a27bb9ff', 'request_id': 'req-6232ae39-fc4b-4840-a06c-d3e4a59795a0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:15.240793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c027fd5054d44b4a52d36a6a27bb9ff', 'request_id': 'req-6232ae39-fc4b-4840-a06c-d3e4a59795a0'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:15.253860 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] The token cache is being invalidate because project 6c027fd5054d44b4a52d36a6a27bb9ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:15.254145 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:15.254772 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-6232ae39-fc4b-4840-a06c-d3e4a59795a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:15.256378 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1428/2886] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:15 2022] DELETE /identity/v3/projects/6c027fd5054d44b4a52d36a6a27bb9ff => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:15.262157 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.262476 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.262753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.281099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:15.282322 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:15.306895 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:15.349882 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1459/2887] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:15.364113 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.364414 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.364707 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.384996 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:15.386293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:15.399138 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1429/2888] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:15.401083 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.411507 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6dbd7e-fb9f-4f69-91f4-9f94141d8b2f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.417692 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.418489 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.419131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.419131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.420394 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:41:15.420710 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.422339 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.435287 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1460/2889] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:41:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 20 06:41:15.440257 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.450926 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a0331-4b94-4f05-ac5e-9ea56d12b802 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.455785 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.456524 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.456897 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.457245 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.458000 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.460029 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.490455 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1430/2890] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:15 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:15.492074 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.502609 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4adbc-c44c-44a8-b36a-e7489fd3082c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.558980 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.559711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.560000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.560280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.561022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.562950 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.587202 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1461/2891] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:15 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:15.588631 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.600062 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ba44e-c8a8-4704-93a9-257acf1a774f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.605484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.606162 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.606454 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.606908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.608406 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.610027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.625038 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1431/2892] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:15.626221 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.636545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed91772-2f3e-4372-9728-ea3a5d115563 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.641370 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e22196-e6dd-4eb9-ba67-eb1319301a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.642098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22196-e6dd-4eb9-ba67-eb1319301a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.642407 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22196-e6dd-4eb9-ba67-eb1319301a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.642673 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e22196-e6dd-4eb9-ba67-eb1319301a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.643429 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e22196-e6dd-4eb9-ba67-eb1319301a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.645056 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e22196-e6dd-4eb9-ba67-eb1319301a4b admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.656070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1462/2893] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:15.660146 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.670974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8919b892-2aa6-4d9f-901e-ce1cc4f88713 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.676915 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.677895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.678183 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.678466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] PATH_INFO: `/v3/projects/523669ec45de4662a2d2bc78117708f6/users/250368ca59e04c91bf67361f3a3caef0/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.694549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=523669ec45de4662a2d2bc78117708f6, user_id=250368ca59e04c91bf67361f3a3caef0, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.697210 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.706477 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:15.714518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1432/2894] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:15 2022] PUT /identity/v3/projects/523669ec45de4662a2d2bc78117708f6/users/250368ca59e04c91bf67361f3a3caef0/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 20 06:41:15.729545 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.729841 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.730131 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.759861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] MFA Rules not processed for user `250368ca59e04c91bf67361f3a3caef0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:15.761025 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:15.830712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1463/2895] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:15.838017 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.839782 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:15.901335 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d498a12-0f17-4c7f-94a0-cea73d733551 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:15.905700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf279736-7d3d-4e22-95a8-7872f4484893 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:15.906363 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279736-7d3d-4e22-95a8-7872f4484893 service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:15.906653 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279736-7d3d-4e22-95a8-7872f4484893 service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:15.906939 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf279736-7d3d-4e22-95a8-7872f4484893 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:15.915879 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf279736-7d3d-4e22-95a8-7872f4484893 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:15.917313 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf279736-7d3d-4e22-95a8-7872f4484893 service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:15.926281 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1433/2896] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:15.945078 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:15.970804 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:16.015773 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8707eb17-3cfa-40d3-8ef1-8e8a507714ce None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:16.020351 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:16.021259 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.021536 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.021812 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] PATH_INFO: `/v3/users/250368ca59e04c91bf67361f3a3caef0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.023499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=250368ca59e04c91bf67361f3a3caef0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:16.025408 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:16.074462 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250368ca59e04c91bf67361f3a3caef0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.081791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250368ca59e04c91bf67361f3a3caef0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.082553 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250368ca59e04c91bf67361f3a3caef0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.083185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250368ca59e04c91bf67361f3a3caef0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.095783 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250368ca59e04c91bf67361f3a3caef0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.109413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1464/2897] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:15 2022] DELETE /identity/v3/users/250368ca59e04c91bf67361f3a3caef0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:16.113927 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7a781-a674-4318-9586-6680c922d071 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.114225 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7a781-a674-4318-9586-6680c922d071 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.114507 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7a781-a674-4318-9586-6680c922d071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.133170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-40d7a781-a674-4318-9586-6680c922d071 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:16.134399 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40d7a781-a674-4318-9586-6680c922d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:16.198658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1434/2898] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:16.208767 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:16.210289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:16.275829 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025903ad-2e5c-4c6d-8ec4-9eeba7ba6233 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:16.279460 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd389bc-f4de-4645-b893-da857c4ed65e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:16.280156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd389bc-f4de-4645-b893-da857c4ed65e service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.280444 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd389bc-f4de-4645-b893-da857c4ed65e service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.280723 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd389bc-f4de-4645-b893-da857c4ed65e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.290311 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd389bc-f4de-4645-b893-da857c4ed65e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:16.291803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd389bc-f4de-4645-b893-da857c4ed65e service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:16.301541 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1465/2899] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:16.594804 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7a781-a674-4318-9586-6680c922d071 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:16.596332 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-40d7a781-a674-4318-9586-6680c922d071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:16.612542 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7a781-a674-4318-9586-6680c922d071 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:16.618316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:16.619338 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.619587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.619949 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] PATH_INFO: `/v3/projects/523669ec45de4662a2d2bc78117708f6` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.621068 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=523669ec45de4662a2d2bc78117708f6)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:16.623633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:16.680272 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '523669ec45de4662a2d2bc78117708f6', 'request_id': 'req-e127235a-9789-41ff-940d-290b0f630d0d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.687521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '523669ec45de4662a2d2bc78117708f6', 'request_id': 'req-e127235a-9789-41ff-940d-290b0f630d0d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.693851 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '523669ec45de4662a2d2bc78117708f6', 'request_id': 'req-e127235a-9789-41ff-940d-290b0f630d0d'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.707175 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] The token cache is being invalidate because project 523669ec45de4662a2d2bc78117708f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:16.707514 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.708170 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-e127235a-9789-41ff-940d-290b0f630d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:16.709752 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1435/2900] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:16 2022] DELETE /identity/v3/projects/523669ec45de4662a2d2bc78117708f6 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:16.714584 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.715051 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.715291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.733824 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:16.738964 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:16.790400 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:16.812261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1466/2901] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:16.821109 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.821426 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.821683 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.841407 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:16.842619 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:16.855740 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1436/2902] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:16.860497 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:16.870289 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e921dc4d-31f5-4748-8d5c-4815a122435c None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:16.876482 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:16.877183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.877472 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.877756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.878493 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:41:16.878807 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:16.880457 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:16.892479 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1467/2903] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:41:16.896755 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:16.906101 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d4edc8-0bd1-4ad7-8918-e17dbc133b08 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:16.913803 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:16.914492 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:16.914793 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:16.915072 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:16.915820 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:16.917694 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:16.932354 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:16.946652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1437/2904] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:16 2022] POST /identity/v3/projects => generated 359 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:16.950846 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:16.960969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80bedb-1552-4e85-925b-93bd9ccf4a3a admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.019915 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.020646 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.020934 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.021210 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.021893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.023860 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.051081 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1468/2905] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:16 2022] POST /identity/v3/users => generated 280 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:17.052178 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:17.062491 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9650f2-3e2f-43b7-bed9-c47601713e73 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.067172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.067868 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.068154 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.068435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.069164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.070759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.084431 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1438/2906] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 20 06:41:17.085655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:17.097619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f37fd8-c0c2-4656-ac2c-bada9a2cb49f admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.102887 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fea64a-96ab-451a-a694-d8a07af37031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.103585 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fea64a-96ab-451a-a694-d8a07af37031 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.103874 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fea64a-96ab-451a-a694-d8a07af37031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.104160 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fea64a-96ab-451a-a694-d8a07af37031 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.104885 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fea64a-96ab-451a-a694-d8a07af37031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.106486 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fea64a-96ab-451a-a694-d8a07af37031 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.117066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1469/2907] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:17.121179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:17.134188 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0563a94c-3a2c-49ad-9361-d82678177062 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.138829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.139778 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.140066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.140344 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] PATH_INFO: `/v3/projects/4ff93b9f658141f5826088e5b2dec117/users/74f67b8849394e468ae9fd73ee7f25d2/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.162293 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ff93b9f658141f5826088e5b2dec117, user_id=74f67b8849394e468ae9fd73ee7f25d2, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.165018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.178630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1439/2908] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:17 2022] PUT /identity/v3/projects/4ff93b9f658141f5826088e5b2dec117/users/74f67b8849394e468ae9fd73ee7f25d2/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:17.188775 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.189179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.189533 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.223414 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] MFA Rules not processed for user `74f67b8849394e468ae9fd73ee7f25d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:17.224962 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:17.300847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1470/2909] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:17 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:17.330870 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:17.332681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:17.392468 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61088390-566b-43d0-bd0e-25382b0f55c2 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.396149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d0f31a-ea0f-46ee-8d4e-c772102a944c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.396825 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0f31a-ea0f-46ee-8d4e-c772102a944c service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.397131 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0f31a-ea0f-46ee-8d4e-c772102a944c service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.397405 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d0f31a-ea0f-46ee-8d4e-c772102a944c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.410783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0f31a-ea0f-46ee-8d4e-c772102a944c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.410783 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d0f31a-ea0f-46ee-8d4e-c772102a944c service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.417857 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1440/2910] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:17.432009 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:17.489252 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:17.512627 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb04db7-74db-46a4-a4f4-bf6d7dd68492 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.517699 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.518579 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.518865 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.519144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] PATH_INFO: `/v3/users/74f67b8849394e468ae9fd73ee7f25d2` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.520520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f67b8849394e468ae9fd73ee7f25d2)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.522405 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.571485 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f67b8849394e468ae9fd73ee7f25d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:17.578677 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f67b8849394e468ae9fd73ee7f25d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:17.579381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f67b8849394e468ae9fd73ee7f25d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:17.580000 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f67b8849394e468ae9fd73ee7f25d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:17.594226 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f67b8849394e468ae9fd73ee7f25d2'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:17.612702 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1471/2911] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:17 2022] DELETE /identity/v3/users/74f67b8849394e468ae9fd73ee7f25d2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:17.617028 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.617518 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.617621 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.634266 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:17.635485 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:17.702149 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1441/2912] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:17.711803 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:17.713722 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:17.784412 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff106d-c9f2-44ba-8e99-8c60fcbbe465 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:17.788748 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0464aa5-66a5-46aa-a1bc-ebe9c7bf31cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:17.789409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0464aa5-66a5-46aa-a1bc-ebe9c7bf31cb service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:17.789758 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0464aa5-66a5-46aa-a1bc-ebe9c7bf31cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:17.790145 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0464aa5-66a5-46aa-a1bc-ebe9c7bf31cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:17.800036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0464aa5-66a5-46aa-a1bc-ebe9c7bf31cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:17.801619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0464aa5-66a5-46aa-a1bc-ebe9c7bf31cb service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:17.812331 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1472/2913] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:18.114989 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.116415 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:18.129427 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a21f09-07fa-440e-9b7d-10f82aa2efc4 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.134684 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.137295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.137295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.137295 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] PATH_INFO: `/v3/projects/4ff93b9f658141f5826088e5b2dec117` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.139656 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ff93b9f658141f5826088e5b2dec117)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.141582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.195356 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ff93b9f658141f5826088e5b2dec117', 'request_id': 'req-fe7397c1-c8b6-449f-b252-1cc5165e0995'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:18.200732 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ff93b9f658141f5826088e5b2dec117', 'request_id': 'req-fe7397c1-c8b6-449f-b252-1cc5165e0995'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:18.202587 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:18.211395 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ff93b9f658141f5826088e5b2dec117', 'request_id': 'req-fe7397c1-c8b6-449f-b252-1cc5165e0995'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:18.225520 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] The token cache is being invalidate because project 4ff93b9f658141f5826088e5b2dec117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:18.225814 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:18.226445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-fe7397c1-c8b6-449f-b252-1cc5165e0995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:18.228991 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1442/2914] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:18 2022] DELETE /identity/v3/projects/4ff93b9f658141f5826088e5b2dec117 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:18.234179 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.234524 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.234809 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.255873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:18.257066 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:18.318655 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1473/2915] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:18.327423 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.329185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.329484 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.349347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:18.350565 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:18.359244 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1443/2916] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:18.363615 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.368679 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:18.375752 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5178b3-9dc5-492f-b478-0bb69d8b371f None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.381558 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.382371 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.382688 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.383120 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.383907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80160) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 20 06:41:18.384223 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.385863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.398266 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1474/2917] 10.222.0.45 () {58 vars in 1152 bytes} [Mon Jun 20 06:41:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 20 06:41:18.402559 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.412150 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08b511-a7af-48c6-95f9-182796ea841f None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.418501 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.419196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.419521 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.420307 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.421018 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.422924 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.451448 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1444/2918] 10.222.0.45 () {60 vars in 1152 bytes} [Mon Jun 20 06:41:18 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:18.455680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.466338 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b872beb-aaaf-4f9f-adfb-c50e4bea85b2 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.530409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.531099 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.531446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.531721 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.532435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.534446 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.557050 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1475/2919] 10.222.0.45 () {60 vars in 1143 bytes} [Mon Jun 20 06:41:18 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 20 06:41:18.561301 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.573916 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69a387-1f40-4840-9d4b-d41f846d3c0e admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.579367 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.580044 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.580329 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.580599 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] PATH_INFO: `/v3/roles` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.581341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.582932 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.594797 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1445/2920] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:18.599435 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.611451 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583cbb3-d8d7-4f62-905e-755f26e3a4a3 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.616194 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e6aa0f-f1d2-4013-85a7-6a0cb636ff07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.616870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aa0f-f1d2-4013-85a7-6a0cb636ff07 admin admin] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.617163 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aa0f-f1d2-4013-85a7-6a0cb636ff07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.617445 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e6aa0f-f1d2-4013-85a7-6a0cb636ff07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.618183 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6aa0f-f1d2-4013-85a7-6a0cb636ff07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.619851 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e6aa0f-f1d2-4013-85a7-6a0cb636ff07 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.634631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1476/2921] 10.222.0.45 () {58 vars in 1121 bytes} [Mon Jun 20 06:41:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 20 06:41:18.635843 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.646372 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eb5469-25c9-4cd6-8b59-eb424592122f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.651341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.652326 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.652685 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.653126 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] PATH_INFO: `/v3/projects/4dc7ed34db294a129524800e020e2927/users/754d51895486408b91743437e4baba03/roles/10f492ea4ccb4aa0b627ead79bc02dea` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.669051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc7ed34db294a129524800e020e2927, user_id=754d51895486408b91743437e4baba03, role_id=10f492ea4ccb4aa0b627ead79bc02dea)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.672348 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.692014 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1446/2922] 10.222.0.45 () {60 vars in 1482 bytes} [Mon Jun 20 06:41:18 2022] PUT /identity/v3/projects/4dc7ed34db294a129524800e020e2927/users/754d51895486408b91743437e4baba03/roles/10f492ea4ccb4aa0b627ead79bc02dea => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:18.703413 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af37be2-4ece-46e9-a1a3-f8555b3611e9 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.703712 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af37be2-4ece-46e9-a1a3-f8555b3611e9 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.703998 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af37be2-4ece-46e9-a1a3-f8555b3611e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.733781 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-5af37be2-4ece-46e9-a1a3-f8555b3611e9 None None] MFA Rules not processed for user `754d51895486408b91743437e4baba03`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:18.735071 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-5af37be2-4ece-46e9-a1a3-f8555b3611e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:18.802791 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1477/2923] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:18.814179 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:18.815941 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:18.880874 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc2950-ff38-4c21-9e57-906c05731593 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:18.884788 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00892de83c5444c7a38b921615b2f5ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:18.885443 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:18.885730 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:18.886009 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:18.905607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:18.907081 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:18.916341 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1447/2924] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:20.741532 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:20.742141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:20.742141 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:20.760305 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] MFA Rules not processed for user `0c0fdef83aed4170b9d0db550daa17d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:20.761515 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:20.827126 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1478/2925] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:28.915676 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=57,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:30.815188 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=96,keystone:INSERT=3 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:32.589153 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:32.590756 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:32.651713 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081706b-3cb8-49be-beb6-f1b9c5389f1a service cinder] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:32.655912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:32.656591 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:32.656881 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:32.657164 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:32.658928 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:32.669392 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:32.671425 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d93c4b-af09-4ba2-89bd-ba445037c477 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:32.679855 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1448/2926] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:41.588428 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:41.677861 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0113a4-b606-48d6-8681-b7eebb96a5df None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:41.682499 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:41.683418 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:41.683698 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:41.683976 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] PATH_INFO: `/v3/users/754d51895486408b91743437e4baba03` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:41.685185 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=754d51895486408b91743437e4baba03)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:41.687693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:41.738384 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754d51895486408b91743437e4baba03'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:41.747084 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754d51895486408b91743437e4baba03'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:41.748549 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754d51895486408b91743437e4baba03'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:41.749265 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754d51895486408b91743437e4baba03'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:41.761164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754d51895486408b91743437e4baba03'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:41.774745 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1479/2927] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:41 2022] DELETE /identity/v3/users/754d51895486408b91743437e4baba03 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:41.779733 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:41.780182 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:41.780700 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:41.798201 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:41.799566 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:41.864051 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1449/2928] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:41.874426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:41.876045 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:41.936487 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d729c4c1-6244-437a-9fa6-cafdab0c5e7c admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:41.940142 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:41.940788 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:41.941070 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:41.941343 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:41.950263 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:41.952309 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:41.961631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1480/2929] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 20 06:41:42.281494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:42.283347 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:42.307444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1104e862-731a-4e83-9d4d-fd76ddb5c28d None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:42.313083 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:42.313976 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:42.314265 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:42.314551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] PATH_INFO: `/v3/projects/4dc7ed34db294a129524800e020e2927` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:42.315945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc7ed34db294a129524800e020e2927)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:42.317945 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:42.370658 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc7ed34db294a129524800e020e2927', 'request_id': 'req-d8967d30-4a01-4855-af29-15642a97bc2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:42.377111 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc7ed34db294a129524800e020e2927', 'request_id': 'req-d8967d30-4a01-4855-af29-15642a97bc2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:42.383667 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc7ed34db294a129524800e020e2927', 'request_id': 'req-d8967d30-4a01-4855-af29-15642a97bc2f'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:42.398666 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] The token cache is being invalidate because project 4dc7ed34db294a129524800e020e2927 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:42.399287 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:42.400058 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:42.401974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1450/2930] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:42 2022] DELETE /identity/v3/projects/4dc7ed34db294a129524800e020e2927 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:44.182069 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:44.183764 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:44.195129 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a45180-0e8f-4151-bea1-078f64adc357 service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:44.212616 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:44.248520 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:44.249171 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:44.250110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:44.250381 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:44.252232 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:44.316941 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:44.318393 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d157f0-e988-435a-8a80-0117e1b1277b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:44.328506 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1481/2931] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:51.845398 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:51.846854 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:51.879895 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:51.917635 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8967d30-4a01-4855-af29-15642a97bc2f admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:51.922273 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:51.923148 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:51.923475 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:51.923759 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] PATH_INFO: `/v3/users/9afdb957fe8f4cf88bcdb14450abce7a` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:51.924947 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afdb957fe8f4cf88bcdb14450abce7a)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:51.926982 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:51.974152 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afdb957fe8f4cf88bcdb14450abce7a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:51.981370 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afdb957fe8f4cf88bcdb14450abce7a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:51.982063 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afdb957fe8f4cf88bcdb14450abce7a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:51.982668 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afdb957fe8f4cf88bcdb14450abce7a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:51.995285 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afdb957fe8f4cf88bcdb14450abce7a'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:52.008535 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1451/2932] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:41:51 2022] DELETE /identity/v3/users/9afdb957fe8f4cf88bcdb14450abce7a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:41:52.014212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:52.014693 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:52.014907 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:52.033098 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] MFA Rules not processed for user `ee104dedf13a4c7aa48858230c010f93`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:41:52.034307 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:52.094440 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1482/2933] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:41:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:41:52.104451 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:52.105858 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:52.167066 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febef3ea-21f3-415b-a11b-19d7c45c9f67 admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:52.171538 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:52.172217 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:52.172596 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:52.172914 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:52.182523 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:52.184024 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:52.194481 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1452/2934] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:41:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:41:52.463481 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:41:52.464911 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:41:52.479523 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b717e4b9-d5a5-44e6-8d1d-0c5a82f7cbea None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:41:52.484345 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:41:52.485253 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:41:52.485542 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:41:52.485837 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] PATH_INFO: `/v3/projects/74a4a77a30cb4b32b481bfd7a56bdba5` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:41:52.487827 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74a4a77a30cb4b32b481bfd7a56bdba5)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:41:52.489753 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:41:52.542909 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74a4a77a30cb4b32b481bfd7a56bdba5', 'request_id': 'req-b86ae0e4-3e36-4127-a848-d5a7031e1798'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:52.549759 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74a4a77a30cb4b32b481bfd7a56bdba5', 'request_id': 'req-b86ae0e4-3e36-4127-a848-d5a7031e1798'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:52.555928 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74a4a77a30cb4b32b481bfd7a56bdba5', 'request_id': 'req-b86ae0e4-3e36-4127-a848-d5a7031e1798'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:52.559535 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:41:52.571348 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] The token cache is being invalidate because project 74a4a77a30cb4b32b481bfd7a56bdba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80160) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:41:52.571643 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:52.572291 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:41:52.573847 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1483/2935] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:41:52 2022] DELETE /identity/v3/projects/74a4a77a30cb4b32b481bfd7a56bdba5 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:42:02.190100 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=67,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:42:02.570498 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=2 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:43:43.610311 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:43:43.611770 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:43.623672 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ac82c-6455-46d1-8188-9fd294536af3 service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:43:43.671577 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:43:43.672215 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:43:43.672497 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:43:43.672771 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:43:43.674483 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:43.742102 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:43:43.743571 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c94dc-1e97-488f-9020-5377dd29f4c0 service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:43:43.754526 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1453/2936] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:43:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:43:46.149997 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:43:46.151392 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:46.170426 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ae0e4-3e36-4127-a848-d5a7031e1798 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:43:46.176665 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:43:46.177368 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:43:46.177661 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:43:46.177943 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:43:46.179977 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:46.194005 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:43:46.195719 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:43:46.209086 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1484/2937] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:43:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:43:47.189318 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] REQUEST_METHOD: `POST` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:43:47.189792 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:43:47.189917 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:43:47.206686 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.auth.core [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] MFA Rules not processed for user `0c0fdef83aed4170b9d0db550daa17d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80161) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:43:47.207993 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:47.262172 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1454/2938] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:43:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:43:47.565756 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:43:47.567323 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:47.579434 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00614c3f-6088-4cf1-a20c-ecf3f51b09c3 service nova] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:43:47.582965 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:43:47.583639 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:43:47.583929 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:43:47.584212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:43:47.585925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:47.595156 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:43:47.596619 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:43:47.604873 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1485/2939] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:43:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:43:48.029008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:43:48.030413 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:48.082299 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=100 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:43:48.103624 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e584b-b257-4d7e-97a7-41ae5eb28164 None None] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:43:48.109156 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:43:48.109812 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:43:48.110106 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:43:48.110385 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:43:48.112158 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:43:48.125021 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:43:48.126582 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:43:48.136435 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1455/2940] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:43:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:43:57.601144 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=12 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:43:58.132240 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=11 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:44:00.839565 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:44:00.854249 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f448767-81b9-4999-acef-07e69dfc562b service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:44:00.859963 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:44:00.860611 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:44:00.860893 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:44:00.861167 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:44:00.862968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:44:00.876310 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:44:00.877925 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf9133-39fa-4bf5-a2f6-fa5781a67783 service nova] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:44:00.890438 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1486/2941] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:44:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:44:04.190745 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:44:04.204008 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793edf6-381a-4848-a6f8-b0b43ccd00e9 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:44:04.209601 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:44:04.210261 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:44:04.210549 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:44:04.210829 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:44:04.212630 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:44:04.225971 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:44:04.227652 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:44:04.239224 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1456/2942] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:44:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:44:10.885200 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=6 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:44:14.234373 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=6 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:45:50.577375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] REQUEST_METHOD: `POST` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:45:50.577870 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:45:50.577968 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:45:50.595947 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.auth.core [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] MFA Rules not processed for user `0c0fdef83aed4170b9d0db550daa17d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80160) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 20 06:45:50.597397 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:45:50.618100 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1487/2943] 10.222.0.45 () {58 vars in 957 bytes} [Mon Jun 20 06:45:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 20 06:45:59.471891 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:45:59.497411 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ff1a9-b02a-4da5-92db-e73e21c3ad46 service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:45:59.503016 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ec34108c1ae24ef699265e0be32e99cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:45:59.503681 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:45:59.503974 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:45:59.504259 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:45:59.506027 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:45:59.516839 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:45:59.518420 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:45:59.527799 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1457/2944] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:45:59 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:46:00.612189 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=9 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:46:09.523715 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=12 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:46:12.639073 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:46:12.678164 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d8e0b-d2a3-46c3-a7cc-bfc38c593d34 None None] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:46:12.683192 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:46:12.683857 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:46:12.684146 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:46:12.684427 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:46:12.686241 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:46:12.709634 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:46:12.711848 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:46:12.721530 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1488/2945] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:46:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:46:22.717325 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=23 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:47:18.360574 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:18.380723 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b336d0-a345-481d-a5fd-436eaffdcf6c service nova] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:18.385020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:18.386346 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:18.386633 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:18.386912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:18.389494 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:47:18.409958 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:18.411445 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:18.419509 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1458/2946] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:47:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:47:28.415802 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=15 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:47:44.580034 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:44.606919 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bbafb7-0160-4053-8802-5dfd032cae8b service placement] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:44.610631 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fcefc528820f4d95b779c8972b46bd0f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:44.611306 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:44.611596 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:44.611875 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:44.613774 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:47:44.696261 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:44.697711 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:44.717072 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1489/2947] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:47:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:47:47.732822 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:47.757800 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488757ab-92eb-49ed-9927-4c61e8056f9a service neutron] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:47.763663 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:47.764551 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] REQUEST_METHOD: `GET` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:47.765030 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:47.765488 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:47.767645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:47:47.783011 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:47.784648 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:47.801560 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1459/2948] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:47:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:47:50.854027 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:50.855747 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:47:50.868064 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c73213-bc79-4eee-b8f0-083099da428d service neutron] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:50.872832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:50.873709 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:50.873999 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:50.874280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] PATH_INFO: `/v3/users/0c0fdef83aed4170b9d0db550daa17d0` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:50.875490 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0fdef83aed4170b9d0db550daa17d0)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:50.877409 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:50.922430 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0fdef83aed4170b9d0db550daa17d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:50.929122 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0fdef83aed4170b9d0db550daa17d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:50.929832 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0fdef83aed4170b9d0db550daa17d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:50.930391 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0fdef83aed4170b9d0db550daa17d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:50.941969 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0fdef83aed4170b9d0db550daa17d0'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:50.954077 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1490/2949] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:47:50 2022] DELETE /identity/v3/users/0c0fdef83aed4170b9d0db550daa17d0 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:47:51.126345 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:51.140061 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89d8ac-5af1-455f-901c-5e73f611db16 service placement] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:51.202185 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:51.203054 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:51.203374 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:51.203662 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] PATH_INFO: `/v3/projects/6a9945a382d749c99a76913dcce97740` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:51.204834 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a9945a382d749c99a76913dcce97740)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:51.206712 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:51.251645 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a9945a382d749c99a76913dcce97740', 'request_id': 'req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.257003 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a9945a382d749c99a76913dcce97740', 'request_id': 'req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.263554 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a9945a382d749c99a76913dcce97740', 'request_id': 'req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.275151 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] The token cache is being invalidate because project 6a9945a382d749c99a76913dcce97740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:47:51.275466 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.276080 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.277592 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1460/2950] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:47:51 2022] DELETE /identity/v3/projects/6a9945a382d749c99a76913dcce97740 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:47:51.281512 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:51.282961 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:47:51.296328 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec5dcf-68ce-42c8-b19a-c3af037e41b9 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:51.314548 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:47:51.353760 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:51.354641 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:51.354930 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:51.355212 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] PATH_INFO: `/v3/users/743fbe23d591471085d43d2aff010d18` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:51.356419 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=743fbe23d591471085d43d2aff010d18)` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:51.358863 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:51.403811 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '743fbe23d591471085d43d2aff010d18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.410680 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '743fbe23d591471085d43d2aff010d18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.411375 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '743fbe23d591471085d43d2aff010d18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.412022 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '743fbe23d591471085d43d2aff010d18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.424238 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.notifications [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '743fbe23d591471085d43d2aff010d18'} {{(pid=80160) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.437190 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1491/2951] 10.222.0.45 () {58 vars in 1223 bytes} [Mon Jun 20 06:47:51 2022] DELETE /identity/v3/users/743fbe23d591471085d43d2aff010d18 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:47:51.616316 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Authenticating user token {{(pid=80161) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:47:51.618020 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.fernet_utils [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80161) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:47:51.634209 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7082c-4166-4e69-9ad9-245d2ceb3cdf admin admin] Validating token access rules against request {{(pid=80161) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:47:51.695005 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee104dedf13a4c7aa48858230c010f93', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b72f99dc88224a27868e009fcc68d646', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80161) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:47:51.695908 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:47:51.696196 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:47:51.696480 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] PATH_INFO: `/v3/projects/e456561d90e9484cab9c3851961821cc` {{(pid=80161) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:47:51.697673 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e456561d90e9484cab9c3851961821cc)` {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:47:51.699607 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] RBAC: Authorization granted {{(pid=80161) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:47:51.722375 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:47:51.749036 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e456561d90e9484cab9c3851961821cc', 'request_id': 'req-cc9adae1-bb1b-4e63-8914-6445e6292237'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.754440 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e456561d90e9484cab9c3851961821cc', 'request_id': 'req-cc9adae1-bb1b-4e63-8914-6445e6292237'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.760444 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e456561d90e9484cab9c3851961821cc', 'request_id': 'req-cc9adae1-bb1b-4e63-8914-6445e6292237'} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.772620 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] The token cache is being invalidate because project e456561d90e9484cab9c3851961821cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80161) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 20 06:47:51.772912 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.773545 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG keystone.notifications [None req-cc9adae1-bb1b-4e63-8914-6445e6292237 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80161) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 20 06:47:51.775071 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: [pid: 80161|app: 0|req: 1461/2952] 10.222.0.45 () {58 vars in 1232 bytes} [Mon Jun 20 06:47:51 2022] DELETE /identity/v3/projects/e456561d90e9484cab9c3851961821cc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 20 06:48:01.433097 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:48:01.769751 dv-cinder-804757-29-iscsi devstack@keystone.service[80161]: DEBUG dbcounter [-] [80161] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80161) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 20 06:51:17.629614 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Authenticating user token {{(pid=80160) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 20 06:51:17.631036 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:51:17.709110 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2362b4-30ef-4e87-a734-da4965e48617 admin admin] Validating token access rules against request {{(pid=80160) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 20 06:51:17.767121 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e93c507849c04762bf311c0c3108bab3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a62eb9ba0184f96ac14a0536d5b646e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80160) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 20 06:51:17.767794 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] REQUEST_METHOD: `GET` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 20 06:51:17.768082 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 20 06:51:17.768367 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80160) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 20 06:51:17.770691 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.fernet_utils [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80160) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 20 06:51:17.781667 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 20 06:51:17.783280 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6dda4b-286d-4903-a01d-c07f127cc0e3 service placement] RBAC: Authorization granted {{(pid=80160) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 20 06:51:17.793002 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: [pid: 80160|app: 0|req: 1492/2953] 10.222.0.45 () {60 vars in 1365 bytes} [Mon Jun 20 06:51:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 20 06:51:27.788768 dv-cinder-804757-29-iscsi devstack@keystone.service[80160]: DEBUG dbcounter [-] [80160] Writing DB stats keystone:SELECT=58 {{(pid=80160) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}